Re: Executing HQL files from JAVA application.

2015-03-21 Thread Steve Howard
There are more elegant ways I am sure, but you could also use a java.io.BufferedReader and read the file content into a string and execute it much as you would a hard coded SQL statement in your class. Sent from my iPad On Mar 21, 2015, at 5:04 AM, Amal Gupta amal.gup...@aexp.com wrote: Hi

Executing HQL files from JAVA application.

2015-03-21 Thread Amal Gupta
Hi Everyone, I am trying to execute a hive *.hql file from a java application. I had tried a couple of ways of doing it through JDBC driver for hive and through spring jdbc template but yet, the only way which was successful for me was to create a runtime process and then execute it. The

Submitting via WebHCat won't put hive log into ATS?

2015-03-21 Thread Xiaoyong Zhu
Hi Hive experts, I know that Hive is writing logs to ATS (application timeline server) when a hive script is executed from the CLI. However, when I try to submit Hive jobs from WebHCat, it seems that it does not write logs into ATS? How could I configure Hive to make the jobs submitted from

RE: Executing HQL files from JAVA application.

2015-03-21 Thread Amal Gupta
Thanks lot Steve for your time and response. I did try that out and was able to execute the basic select statements. But the concern I had is that the queries contained in the hql files are complicated queries with joins, grouping etc and they also do some insert operations. Essentially, I was

RE: Executing HQL files from JAVA application.

2015-03-21 Thread Mich Talebzadeh
Hi Amal, Do you have hiveserver2 running? You can use beeline to execute the query outside of JAVA beeline -u jdbc:hive2://rhes564:10010/default org.apache.hive.jdbc.HiveDriver -n hduser -p ' -f ./create_index_on_t.sql And the output shows there as well. scan complete in

RE: Executing HQL files from JAVA application.

2015-03-21 Thread Amal Gupta
Hi Mich, Thank you for your response. I was not aware of beeline. I have now included this in my app and this looks a much better solution going forward. In the last couple of hours I have tried to work with beeline but have been facing some issues. 1. I was able to run on the remote

RE: Executing HQL files from JAVA application.

2015-03-21 Thread Mich Talebzadeh
Hi Amal; Me coming from relational database (Oracle, Sybase) background J always expect that a DDL statement like DROP TABLE has to run in its own transaction and cannot be combined with a DML statement. Now I suspect that when you run the command DROP TABLE IF EXIASTS TABLE_NAME; like

How to clean up a table for which the underlying hdfs file no longer exists

2015-03-21 Thread Stephen Boesch
There is a hive table for which the metadata points to a non-existing hdfs file. Simply calling drop table mytable results in: Failed to load metadata for table: db.mytable Caused by TAbleLoadingException: Failed to load metadata for table: db.mytable File does not exist: hdfs://

RE: Executing HQL files from JAVA application.

2015-03-21 Thread Amal Gupta
Hi Mich, :) A coincidence. Even I am new to hive. My test script which I am trying to execute contains a drop and a create statement. Script :- use test_db; DROP TABLE IF EXISTS demoHiveTable; CREATE EXTERNAL TABLE demoHiveTable ( demoId string, demoName string ) ROW FORMAT DELIMITED FIELDS

Re: How to clean up a table for which the underlying hdfs file no longer exists

2015-03-21 Thread Daniel Haviv
You can (as a workaround) just create it's directory and then drop it Daniel On 22 במרץ 2015, at 04:15, Stephen Boesch java...@gmail.com wrote: There is a hive table for which the metadata points to a non-existing hdfs file. Simply calling drop table mytable results in:

Re: How to clean up a table for which the underlying hdfs file no longer exists

2015-03-21 Thread Daniel Haviv
You can also use ALTER TABLE table SET TBLPROPERTIES('EXTERNAL'='TRUE') And then drop it Daniel On 22 במרץ 2015, at 04:15, Stephen Boesch java...@gmail.com wrote: There is a hive table for which the metadata points to a non-existing hdfs file. Simply calling drop table mytable