Hi I tried truncating the table(ensuring no data files are present in the warehouse directory) and then used load data inpath with/without overwrite option but nothing worked.
Hive directory is owned by hive user but my existing user which is a application user owns the complete access on hive warehouse directory as I can easily delete the files from hive warehouse dir. Will check the HS2 logs for further debugging. On Oct 21, 2015 7:38 AM, "Matthew Bigelow" <[email protected]> wrote: > This is a shot in the dark but... have you tried load data inpath into a > fresh table. I have not had issues with this commmand and HS2. I havent > done it with Overwrite though. Just load data inpath 'hdfs-file' onto > table. > > I dont see it in the exception but permission issues can cause it to fail > the movetask portion of this command. The user running HS2 need full > access to the hive warehouse directory. Is the table managed by hive? > On Oct 20, 2015 6:54 PM, "Vineet Mishra" <[email protected]> wrote: > >> Hi, >> >> I already stated my requirement since the files are generating on hdfs >> and i want to load the file from hdfs to hive instead of local file system. >> >> Moreover in production like environment one just have port access to HS2 >> node and no direct access to machine, in that particular case how do we >> expect load local to be the only solution. >> >> I have been doing the load inpath using HS1 but due to concurrency and gc >> issue I am moving to HS2 but if its just dependent on load local then its >> really strange. >> >> Thanks! >> On Oct 21, 2015 6:22 AM, "[email protected]" <[email protected]> wrote: >> >>> Place the file on the hiveserver2 host local filesystem . Then try >>> load data local inpath >>> >>> ------------------------------ >>> [email protected] >>> >>> >>> *From:* Vineet Mishra <[email protected]> >>> *Date:* 2015-10-21 03:05 >>> *To:* user <[email protected]> >>> *CC:* cdh-user <[email protected]> >>> *Subject:* Re: HiveServer2 load data inpath fails >>> Any update around this? >>> >>> 0: jdbc:hive2://hadoop-hs2:10000> load data inpath '/data/my_table/' >>> overwrite into table mydb.mytable; >>> Error: Error while processing statement: FAILED: Execution Error, return >>> code 1 from org.apache.hadoop.hive.ql.exec.MoveTask (state=08S01,code=1) >>> >>> I couldn't get any possible solution for this. Does it mean HS2 is >>> incapable of supporting load data or there's no JIRA open for it? >>> >>> Version : 0.13.1-cdh5.3.2 >>> >>> On Tue, Oct 20, 2015 at 10:54 PM, Vineet Mishra <[email protected]> >>> wrote: >>> >>>> My data files resides on hdfs and for the given command I guess it will >>>> point to local filesystem instead of hdfs. >>>> On Oct 20, 2015 10:25 PM, "Vikas Srivastava" <[email protected]> >>>> wrote: >>>> >>>>> You Need to use “load data local inpath” >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> *From:* Vineet Mishra [mailto:[email protected]] >>>>> *Sent:* Tuesday, October 20, 2015 6:08 PM >>>>> *To:* [email protected]; [email protected] to >>>>> *Subject:* HiveServer2 load data inpath fails >>>>> >>>>> >>>>> >>>>> Hi All, >>>>> >>>>> I am trying to run load data inpath to update/refresh my table data in >>>>> hive. I am using HiveServer2 to make jdbc connection and run the following >>>>> command >>>>> >>>>> load data inpath 'path_to_file_on_hdfs' overwrite into table tablename; >>>>> >>>>> Eventually this query works fine with HiveServer1 through thrift >>>>> connection but fails through jdbc connection on HS2, stack trace mentioned >>>>> below. >>>>> >>>>> java.sql.SQLException: Error while processing statement: FAILED: >>>>> Execution Error, return code 1 from >>>>> org.apache.hadoop.hive.ql.exec.MoveTask >>>>> >>>>> at org.apache.hive.jdbc.HiveStatement.execute(HiveStatement.java:277) >>>>> >>>>> at >>>>> com.sd.dwh.sc.tungsten.misc.CommonFunction.executeQuery(CommonFunction.java:813) >>>>> >>>>> at >>>>> com.sd.dwh.sc.tungsten.misc.CommonFunction.dataLoadFromFile(CommonFunction.java:836) >>>>> >>>>> at >>>>> com.sd.dwh.sc.tungsten.custom.JDBCMaterialize.merge(JDBCMaterialize.java:105) >>>>> >>>>> at >>>>> com.sd.dwh.sc.tungsten.custom.JDBCMaterialize.run(JDBCMaterialize.java:54) >>>>> >>>>> at >>>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) >>>>> >>>>> at >>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) >>>>> >>>>> at java.lang.Thread.run(Thread.java:745) >>>>> >>>>> Any suggestions would be highly appreciated. >>>>> >>>>> URGENT CALL. >>>>> >>>>> Thanks! >>>>> >>>> >>> -- >> >> --- >> You received this message because you are subscribed to the Google Groups >> "CDH Users" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> For more options, visit https://groups.google.com/a/cloudera.org/d/optout >> . >> > -- > > --- > You received this message because you are subscribed to the Google Groups > "CDH Users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > For more options, visit https://groups.google.com/a/cloudera.org/d/optout. >
