Hi Vineet, Were you able to find anything in HS2 log?
I was just able to run 'load data inpath' with hive ver. 0.14 without any issue. My env is hive.execution.engine=tez though. Also Based on your error message, it may be worth checking what value is set for datanucleus.connectionPoolingType. I have mine set to datanucleus.connectionPoolingType=BONECP. Cheers, On Tue, Oct 20, 2015 at 8:45 PM, Vineet Mishra <[email protected]> wrote: > Hi > > I tried truncating the table(ensuring no data files are present in the > warehouse directory) and then used load data inpath with/without overwrite > option but nothing worked. > > Hive directory is owned by hive user but my existing user which is a > application user owns the complete access on hive warehouse directory as I > can easily delete the files from hive warehouse dir. > > Will check the HS2 logs for further debugging. > On Oct 21, 2015 7:38 AM, "Matthew Bigelow" <[email protected]> > wrote: > >> This is a shot in the dark but... have you tried load data inpath into a >> fresh table. I have not had issues with this commmand and HS2. I havent >> done it with Overwrite though. Just load data inpath 'hdfs-file' onto >> table. >> >> I dont see it in the exception but permission issues can cause it to fail >> the movetask portion of this command. The user running HS2 need full >> access to the hive warehouse directory. Is the table managed by hive? >> On Oct 20, 2015 6:54 PM, "Vineet Mishra" <[email protected]> wrote: >> >>> Hi, >>> >>> I already stated my requirement since the files are generating on hdfs >>> and i want to load the file from hdfs to hive instead of local file system. >>> >>> Moreover in production like environment one just have port access to HS2 >>> node and no direct access to machine, in that particular case how do we >>> expect load local to be the only solution. >>> >>> I have been doing the load inpath using HS1 but due to concurrency and >>> gc issue I am moving to HS2 but if its just dependent on load local then >>> its really strange. >>> >>> Thanks! >>> On Oct 21, 2015 6:22 AM, "[email protected]" <[email protected]> >>> wrote: >>> >>>> Place the file on the hiveserver2 host local filesystem . Then try >>>> load data local inpath >>>> >>>> ------------------------------ >>>> [email protected] >>>> >>>> >>>> *From:* Vineet Mishra <[email protected]> >>>> *Date:* 2015-10-21 03:05 >>>> *To:* user <[email protected]> >>>> *CC:* cdh-user <[email protected]> >>>> *Subject:* Re: HiveServer2 load data inpath fails >>>> Any update around this? >>>> >>>> 0: jdbc:hive2://hadoop-hs2:10000> load data inpath '/data/my_table/' >>>> overwrite into table mydb.mytable; >>>> Error: Error while processing statement: FAILED: Execution Error, >>>> return code 1 from org.apache.hadoop.hive.ql.exec.MoveTask >>>> (state=08S01,code=1) >>>> >>>> I couldn't get any possible solution for this. Does it mean HS2 is >>>> incapable of supporting load data or there's no JIRA open for it? >>>> >>>> Version : 0.13.1-cdh5.3.2 >>>> >>>> On Tue, Oct 20, 2015 at 10:54 PM, Vineet Mishra <[email protected] >>>> > wrote: >>>> >>>>> My data files resides on hdfs and for the given command I guess it >>>>> will point to local filesystem instead of hdfs. >>>>> On Oct 20, 2015 10:25 PM, "Vikas Srivastava" <[email protected]> >>>>> wrote: >>>>> >>>>>> You Need to use “load data local inpath” >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> *From:* Vineet Mishra [mailto:[email protected]] >>>>>> *Sent:* Tuesday, October 20, 2015 6:08 PM >>>>>> *To:* [email protected]; [email protected] to >>>>>> *Subject:* HiveServer2 load data inpath fails >>>>>> >>>>>> >>>>>> >>>>>> Hi All, >>>>>> >>>>>> I am trying to run load data inpath to update/refresh my table data >>>>>> in hive. I am using HiveServer2 to make jdbc connection and run the >>>>>> following command >>>>>> >>>>>> load data inpath 'path_to_file_on_hdfs' overwrite into table >>>>>> tablename; >>>>>> >>>>>> Eventually this query works fine with HiveServer1 through thrift >>>>>> connection but fails through jdbc connection on HS2, stack trace >>>>>> mentioned >>>>>> below. >>>>>> >>>>>> java.sql.SQLException: Error while processing statement: FAILED: >>>>>> Execution Error, return code 1 from >>>>>> org.apache.hadoop.hive.ql.exec.MoveTask >>>>>> >>>>>> at org.apache.hive.jdbc.HiveStatement.execute(HiveStatement.java:277) >>>>>> >>>>>> at >>>>>> com.sd.dwh.sc.tungsten.misc.CommonFunction.executeQuery(CommonFunction.java:813) >>>>>> >>>>>> at >>>>>> com.sd.dwh.sc.tungsten.misc.CommonFunction.dataLoadFromFile(CommonFunction.java:836) >>>>>> >>>>>> at >>>>>> com.sd.dwh.sc.tungsten.custom.JDBCMaterialize.merge(JDBCMaterialize.java:105) >>>>>> >>>>>> at >>>>>> com.sd.dwh.sc.tungsten.custom.JDBCMaterialize.run(JDBCMaterialize.java:54) >>>>>> >>>>>> at >>>>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) >>>>>> >>>>>> at >>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) >>>>>> >>>>>> at java.lang.Thread.run(Thread.java:745) >>>>>> >>>>>> Any suggestions would be highly appreciated. >>>>>> >>>>>> URGENT CALL. >>>>>> >>>>>> Thanks! >>>>>> >>>>> >>>> -- >>> >>> --- >>> You received this message because you are subscribed to the Google >>> Groups "CDH Users" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> For more options, visit >>> https://groups.google.com/a/cloudera.org/d/optout. >>> >> -- >> >> --- >> You received this message because you are subscribed to the Google Groups >> "CDH Users" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> For more options, visit https://groups.google.com/a/cloudera.org/d/optout >> . >> > -- Takahiko Saito
