Hive Installation Issue
Hi I'm trying to install HIVE on Hadoop version 1.0.2. I use hive-0.11.0.tar.gz version. Hive successfully gets installed and when i try to run a HIVE command, say SHOW TABLES; i get the following error, hive show tables; Exception in thread main java.lang.NoSuchFieldError: type at org.apache.hadoop.hive.ql.parse.HiveLexer.mKW_SHOW(HiveLexer.java:1305) at org.apache.hadoop.hive.ql.parse.HiveLexer.mTokens(HiveLexer.java:6439) at org.antlr.runtime.Lexer.nextToken(Lexer.java:84) at org.antlr.runtime.CommonTokenStream.fillBuffer(CommonTokenStream.java:95) at org.antlr.runtime.CommonTokenStream.LT(CommonTokenStream.java:238) at org.apache.hadoop.hive.ql.parse.HiveParser.statement(HiveParser.java:573) at org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:439) at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:416) at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:335) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:893) at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:259) at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:216) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:412) at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:755) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:613) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:616) at org.apache.hadoop.util.RunJar.main(RunJar.java:156) Also there is no any other antlr jar in the class path except for the one provided by HIVE. I do not know where exactly is the issue. Please suggest. Thanks Ranjitha. ::DISCLAIMER:: The contents of this e-mail and any attachment(s) are confidential and intended for the named recipient(s) only. E-mail transmission is not guaranteed to be secure or error-free as information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or may contain viruses in transmission. The e mail and its contents (with or without referred errors) shall therefore not attach any liability on the originator or HCL or its affiliates. Views or opinions, if any, presented in this email are solely those of the author and may not necessarily reflect the views or opinions of HCL or its affiliates. Any form of reproduction, dissemination, copying, disclosure, modification, distribution and / or publication of this message without the prior written consent of authorized representative of HCL is strictly prohibited. If you have received this email in error please delete it and notify the sender immediately. Before opening any email and/or attachments, please check them for viruses and other defects.
Re: Hive Installation Issue
Its not just the antlr, there must be no conflicting jars on HADOOP_CLASSPATH versus that required by your hive version. Thanks Rekha From: Ranjitha Chandrashekar ranjitha...@hcl.commailto:ranjitha...@hcl.com Reply-To: user@hive.apache.orgmailto:user@hive.apache.org user@hive.apache.orgmailto:user@hive.apache.org Date: Friday 5 July 2013 2:52 PM To: user@hive.apache.orgmailto:user@hive.apache.org user@hive.apache.orgmailto:user@hive.apache.org Subject: Hive Installation Issue Hi I’m trying to install HIVE on Hadoop version 1.0.2. I use hive-0.11.0.tar.gz version. Hive successfully gets installed and when i try to run a HIVE command, say SHOW TABLES; i get the following error, hive show tables; Exception in thread main java.lang.NoSuchFieldError: type at org.apache.hadoop.hive.ql.parse.HiveLexer.mKW_SHOW(HiveLexer.java:1305) at org.apache.hadoop.hive.ql.parse.HiveLexer.mTokens(HiveLexer.java:6439) at org.antlr.runtime.Lexer.nextToken(Lexer.java:84) at org.antlr.runtime.CommonTokenStream.fillBuffer(CommonTokenStream.java:95) at org.antlr.runtime.CommonTokenStream.LT(CommonTokenStream.java:238) at org.apache.hadoop.hive.ql.parse.HiveParser.statement(HiveParser.java:573) at org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:439) at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:416) at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:335) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:893) at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:259) at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:216) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:412) at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:755) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:613) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:616) at org.apache.hadoop.util.RunJar.main(RunJar.java:156) Also there is no any other antlr jar in the class path except for the one provided by HIVE. I do not know where exactly is the issue. Please suggest. Thanks Ranjitha. ::DISCLAIMER:: The contents of this e-mail and any attachment(s) are confidential and intended for the named recipient(s) only. E-mail transmission is not guaranteed to be secure or error-free as information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or may contain viruses in transmission. The e mail and its contents (with or without referred errors) shall therefore not attach any liability on the originator or HCL or its affiliates. Views or opinions, if any, presented in this email are solely those of the author and may not necessarily reflect the views or opinions of HCL or its affiliates. Any form of reproduction, dissemination, copying, disclosure, modification, distribution and / or publication of this message without the prior written consent of authorized representative of HCL is strictly prohibited. If you have received this email in error please delete it and notify the sender immediately. Before opening any email and/or attachments, please check them for viruses and other defects.
RE: Hive Installation Issue
Hi Rekha Thanks for the quick response. Figured out the issue. This was because of the conflicting antlr jar. This is caused due to wrong resolution of antlr runtime. Instead of resolving from HIVE_HOME/lib/antlr*jar - it was resolving from HADOOP_HOME/lib/mahout-examples-0.7-job.jar The conflicting antlr jar is removed from the classpath and it works fine now. Thanks Ranjitha. From: Joshi, Rekha [mailto:rekha_jo...@intuit.com] Sent: 05 July 2013 16:18 To: user@hive.apache.org Subject: Re: Hive Installation Issue Its not just the antlr, there must be no conflicting jars on HADOOP_CLASSPATH versus that required by your hive version. Thanks Rekha From: Ranjitha Chandrashekar ranjitha...@hcl.commailto:ranjitha...@hcl.com Reply-To: user@hive.apache.orgmailto:user@hive.apache.org user@hive.apache.orgmailto:user@hive.apache.org Date: Friday 5 July 2013 2:52 PM To: user@hive.apache.orgmailto:user@hive.apache.org user@hive.apache.orgmailto:user@hive.apache.org Subject: Hive Installation Issue Hi I'm trying to install HIVE on Hadoop version 1.0.2. I use hive-0.11.0.tar.gz version. Hive successfully gets installed and when i try to run a HIVE command, say SHOW TABLES; i get the following error, hive show tables; Exception in thread main java.lang.NoSuchFieldError: type at org.apache.hadoop.hive.ql.parse.HiveLexer.mKW_SHOW(HiveLexer.java:1305) at org.apache.hadoop.hive.ql.parse.HiveLexer.mTokens(HiveLexer.java:6439) at org.antlr.runtime.Lexer.nextToken(Lexer.java:84) at org.antlr.runtime.CommonTokenStream.fillBuffer(CommonTokenStream.java:95) at org.antlr.runtime.CommonTokenStream.LT(CommonTokenStream.java:238) at org.apache.hadoop.hive.ql.parse.HiveParser.statement(HiveParser.java:573) at org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:439) at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:416) at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:335) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:893) at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:259) at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:216) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:412) at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:755) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:613) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:616) at org.apache.hadoop.util.RunJar.main(RunJar.java:156) Also there is no any other antlr jar in the class path except for the one provided by HIVE. I do not know where exactly is the issue. Please suggest. Thanks Ranjitha. ::DISCLAIMER:: The contents of this e-mail and any attachment(s) are confidential and intended for the named recipient(s) only. E-mail transmission is not guaranteed to be secure or error-free as information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or may contain viruses in transmission. The e mail and its contents (with or without referred errors) shall therefore not attach any liability on the originator or HCL or its affiliates. Views or opinions, if any, presented in this email are solely those of the author and may not necessarily reflect the views or opinions of HCL or its affiliates. Any form of reproduction, dissemination, copying, disclosure, modification, distribution and / or publication of this message without the prior written consent of authorized representative of HCL is strictly prohibited. If you have received this email in error please delete it and notify the sender immediately. Before opening any email and/or attachments, please check them for viruses and other defects.
Re: Loading a flat file + one additional field to a Hive table
Raj, You should dump the data in a temp table first and then move the data into final table with select query. Select date(), c1,c2. From temp table. Reason: we should avoid custom operation in load unless it is necessary. Sent via Rocket from my HTC - Reply message - From: Raj Hadoop hadoop...@yahoo.com To: Hive user@hive.apache.org Subject: Loading a flat file + one additional field to a Hive table Date: Fri, Jul 5, 2013 10:30 PM Hi, Can any one please suggest the best way to do the following in Hive? Load 'todays date stamp' + ALL FIELDS C1,C2,C3,C4 IN A FILE F1 to a Hive table T1 ( D1,C1,C2,C3,C4) Can the following command be modified in some way to acheive the above hive load data local inpath '/software/home/hadoop/dat_files/' into table T1; My requirement is to append a date stamp to a Web log file and then load it to Hive table. Thanks, Raj
Re: Loading a flat file + one additional field to a Hive table
How about this ?
Assume you have a log file called
oompaloompa.log
TIMESTAMP=$(date +%Y_%m_%d_T%H_%M_%S);mv oompaloopa.log
oompaloopa.log.${TIMESTAMP};cat oompaloopa.log.${TIMESTAMP}| hdfs dfs -put -
/user/sasubramanian/oompaloopa.log.${TIMESTAMP}
This will directly put the file on HDFS and u can put it to the LOCATION
specified by your HIVE TABLE definition
sanjay
From: manishbh...@rocketmail.commailto:manishbh...@rocketmail.com
manishbh...@rocketmail.commailto:manishbh...@rocketmail.com
Reply-To: user@hive.apache.orgmailto:user@hive.apache.org
user@hive.apache.orgmailto:user@hive.apache.org
Date: Friday, July 5, 2013 10:39 AM
To: Raj Hadoop hadoop...@yahoo.commailto:hadoop...@yahoo.com, Hive
user@hive.apache.orgmailto:user@hive.apache.org
Subject: Re: Loading a flat file + one additional field to a Hive table
Raj,
You should dump the data in a temp table first and then move the data into
final table with select query.
Select date(), c1,c2. From temp table.
Reason: we should avoid custom operation in load unless it is necessary.
Sent via Rocket from my HTC
- Reply message -
From: Raj Hadoop hadoop...@yahoo.commailto:hadoop...@yahoo.com
To: Hive user@hive.apache.orgmailto:user@hive.apache.org
Subject: Loading a flat file + one additional field to a Hive table
Date: Fri, Jul 5, 2013 10:30 PM
Hi,
Can any one please suggest the best way to do the following in Hive?
Load 'todays date stamp' + ALL FIELDS C1,C2,C3,C4 IN A FILE F1 to a Hive
table T1 ( D1,C1,C2,C3,C4)
Can the following command be modified in some way to acheive the above
hive load data local inpath '/software/home/hadoop/dat_files/' into table
T1;
My requirement is to append a date stamp to a Web log file and then load it to
Hive table.
Thanks,
Raj
CONFIDENTIALITY NOTICE
==
This email message and any attachments are for the exclusive use of the
intended recipient(s) and may contain confidential and privileged information.
Any unauthorized review, use, disclosure or distribution is prohibited. If you
are not the intended recipient, please contact the sender by reply email and
destroy all copies of the original message along with any attachments, from
your computer system. If you are the intended recipient, please be advised that
the content of this message is subject to access, review and disclosure by the
sender's Email System Administrator.
Select columns listed in another table
Hi Hive Gurus, I have a table with a bunch of columns and another table with a list of columns names coming from the first table. What I want to do is to generate a 2-column table that contains column-value pairs from the 1st table but only for columns defined in the second table. In other words, I have the following tables. Table 1:id as string,col1 as string,col2 as string,...colN as string, Table 2:colname as string I'd like to generate a table with the following schema. Table 3:id as string,colname as string,colvalue as string It is relatively straightforward to do this in two steps. We can run a select query on the 2nd table (column names) to get the column list, and generate another Hive query based on this list, but this approach needs to run two separate queries. Are there more efficient ways of achieving the same? Thanks,Sha Liu
