Could you provide your oozie command? It should run as the user who submitted the job.
On Mon, Apr 13, 2015 at 5:40 AM, Vishwakarma, Chhaya < [email protected]> wrote: > I tried –doas <username> no luck with that. > > > > Below is the error log > > > > 31075 [Thread-35] INFO org.apache.sqoop.hive.HiveImport - Loading data > to table ecsdashboard.test_oozie > > 31175 [Thread-35] INFO org.apache.sqoop.hive.HiveImport - Failed with > exception Unable to move source hdfs://user/qjdht93/test/_SUCCESS to > destination hdfs://apps/hive/warehouse/ecsdashboard.db/test_oozie/_SUCCESS > > 31176 [Thread-35] INFO org.apache.sqoop.hive.HiveImport - FAILED: > Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MoveTask > > 31728 [main] ERROR org.apache.sqoop.tool.ImportTool - Encountered > IOException running import job: java.io.IOException: Hive exited with > status 1 > > at > org.apache.sqoop.hive.HiveImport.executeExternalHiveScript(HiveImport.java:385) > > at org.apache.sqoop.hive.HiveImport.executeScript(HiveImport.java:335) > > at org.apache.sqoop.hive.HiveImport.importTable(HiveImport.java:239) > > at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:425) > > at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:506) > > at org.apache.sqoop.Sqoop.run(Sqoop.java:147) > > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) > > at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183) > > at org.apache.sqoop.Sqoop.runTool(Sqoop.java:222) > > at org.apache.sqoop.Sqoop.runTool(Sqoop.java:231) > > at org.apache.sqoop.Sqoop.main(Sqoop.java:240) > > at org.apache.oozie.action.hadoop.SqoopMain.runSqoopJob(SqoopMain.java:206) > > at org.apache.oozie.action.hadoop.SqoopMain.run(SqoopMain.java:174) > > at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:38) > > at org.apache.oozie.action.hadoop.SqoopMain.main(SqoopMain.java:45) > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > > at java.lang.reflect.Method.invoke(Method.java:606) > > at > org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:225) > > at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54) > > at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:430) > > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:342) > > at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168) > > at java.security.AccessController.doPrivileged(Native Method) > > at javax.security.auth.Subject.doAs(Subject.java:415) > > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1594) > > at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163) > > > > > > *From:* Abraham Elmahrek [mailto:[email protected]] > *Sent:* Friday, April 10, 2015 1:46 AM > > *To:* [email protected] > *Subject:* Re: Error in Sqoop workflow using Oozie > > > > I don't fully understand. Do you have the full error? > > > > On Thu, Apr 9, 2015 at 7:40 AM, Vishwakarma, Chhaya < > [email protected]> wrote: > > Thanks Abe > Permissions for jars is r-x for "others" user then too its giving > permission denied error > > Can you suggest something > Sent from my android device. > > > > -----Original Message----- > From: Abraham Elmahrek <[email protected]> > To: "[email protected]" <[email protected]> > Sent: Wed, 08 Apr 2015 8:57 pm > Subject: Re: Error in Sqoop workflow using Oozie > > Not sure. Could be a yarn configuration thing. Sqoop shouldn't be > explicitly creating a directories there unless you're the yarn user. > > > > On Wed, Apr 8, 2015 at 4:15 AM, Vishwakarma, Chhaya < > [email protected]> wrote: > > > > Created /user/yarn directory but why it is referring to this directory ? > > I have received a user ID from Hadoop admin that I’m using for running > Sqoop jobs > > > > Thanks , > > > > > > *From:* Abraham Elmahrek [mailto:[email protected]] > *Sent:* Tuesday, April 07, 2015 11:23 PM > > > *To:* [email protected] > *Subject:* Re: Error in Sqoop workflow using Oozie > > > > Seems like you need to create the "/user/yarn" directory. Also, what user > are you running sqoop as? > > > > On Tue, Apr 7, 2015 at 3:21 AM, Vishwakarma, Chhaya < > [email protected]> wrote: > > Thanks Abe, > > > > Below is the Error I'm getting in Map Reduce logs > > > > error: error reading /usr/lib/hadoop/lib/smore.jar; > /usr/lib/hadoop/lib/smore.jar (Permission denied) > > error: error reading /usr/lib/hadoop/lib/janusclient.jar; > /usr/lib/hadoop/lib/janusclient .jar (Permission denied) > > error: error reading /usr/lib/hadoop/lib/aster- networking.jar; > /usr/lib/hadoop/lib/aster-networking.jar (Permission denied) > > error: error reading /usr/lib/hadoop/lib/adfs-api-loader.jar; > /usr/lib/hadoop/lib/adfs-api-loader.jar (Permission denied) > > error: error reading /usr/lib/hadoop/lib/aftp.jar; > /usr/lib/hadoop/lib/aftp.jar (Permission denied) > > error: error reading /usr/lib/hadoop/lib/adfs.jar; > /usr/lib/hadoop/lib/adfs.jar (Permission denied) > > Note: /tmp/sqoop > yarn/compile/f3aca99f37fa19e505b47bab1499bf5d/EXT_EVNT_ERR_ARC.java uses or > overrides a deprecated API. > > Note: Recompile with -Xlint:deprecation for details. > > Intercepting System.exit(1) > > Failing Oozie Launcher, Main class > [org.apache.oozie.action.hadoop.SqoopMain], exit code [1] > > che.sqoop.hive.HiveImport - **Caused by: java.io.FileNotFoundException: File > does not exist: hdfs:/user/yarn**` > > Please help > > ------------------------------ > > *From:* Abraham Elmahrek [[email protected]] > *Sent:* Tuesday, April 07, 2015 2:12 AM > *To:* [email protected] > *Subject:* Re: Error in Sqoop workflow using Oozie > > A couple of thoughts: > > 1. Oozie will run the sqoop job as the user oozie possibly. So > "hive-site.xml" won't be picked up. Try putting it in a shared location and > adding a symlink: <file>/tmp/hive-site.xml#hive-site.xml</file>. The same > is true for terajdbc4.jar. > 2. The oozie launcher task logs should have more information. It will > be easier to debug if you drill down into the mapreduce job launching > Sqoop. > > -Abe > > > > On Mon, Apr 6, 2015 at 7:40 AM, Vishwakarma, Chhaya < > [email protected]> wrote: > > I have written a Sqoop import script to import data from teradata to hive. > It's working fine when I run it from command line ,but when I put it in > Oozie workflow and try to execute through oozie I get the below error > > > > > > 2015-04-02 08:50:55,440 INFO ActionEndXCommand:539 - USER[qjdht93] > GROUP[-TOKEN[] APP[sqoop-shell-wf] > JOB[0000069-150114201015959-oozie-oozi-W] > ACTION[0000069-150114201015959-oozie-oozi-W@sqoop-shell] end executor for > wf action 0000069-150114201015959-oozie-oozi-W with wf job > 0000069-150114201015959-oozie-oozi-W > > 2015-04-02 08:50:55,459 INFO ActionEndXCommand:539 - USER[qjdht93] > GROUP[-] TOKEN[] APP[sqoop-shell-wf] > JOB[0000069-150114201015959-oozie-oozi-W] > ACTION[0000069-150114201015959-oozie-oozi-W@sqoop-shell] ERROR is > considered as FAILED for SLA > > 2015-04-02 08:50:55,505 INFO ActionStartXCommand:539 - USER[qjdht93] > GROUP[-] TOKEN[] APP[sqoop-shell-wf] > JOB[0000069-150114201015959-oozie-oozi-W] > ACTION[0000069-150114201015959-oozie-oozi-W@fail] Start action > [0000069-150114201015959-oozie-oozi-W@fail] with user-retry state : > userRetryCount [0], userRetryMax [0], userRetryInterval [10] > > 2015-04-02 08:50:55,505 WARN ActionStartXCommand:542 - USER[qjdht93] > GROUP[-] TOKEN[] APP[sqoop-shell-wf] > JOB[0000069-150114201015959-oozie-oozi-W] > ACTION[0000069-150114201015959-oozie-oozi-W@fail] > [***0000069-150114201015959-oozie-oozi-W@fail***]Action status=DONE > > 2015-04-02 08:50:55,505 WARN ActionStartXCommand:542 - USER[qjdht93] > GROUP[-] TOKEN[] APP[sqoop-shell-wf] > JOB[0000069-150114201015959-oozie-oozi-W] > ACTION[0000069-150114201015959-oozie-oozi-W@fail] > [***0000069-150114201015959-oozie-oozi-W@fail***]Action updated in DB! > > 2015-04-02 08:50:55,522 INFO ActionEndXCommand:539 - USER[qjdht93] > GROUP[-] TOKEN[] APP[sqoop-shell-wf] > JOB[0000069-150114201015959-oozie-oozi-W] > ACTION[0000069-150114201015959-oozie-oozi-W@fail] end executor for wf > action 0000069-150114201015959-oozie-oozi-W with wf job > 0000069-150114201015959-oozie-oozi-W > > 2015-04-02 08:50:55,556 WARN CoordActionUpdateXCommand:542 - > USER[qjdht93] GROUP[-] TOKEN[] APP[sqoop-shell-wf] > JOB[0000069-150114201015959-oozie-oozi-W] ACTION[-] **E1100: Command > precondition does not hold before execution, [, coord action is null], > Error Code: E1100**` > > > > Below is my workflow.xml > > <workflow-app name="sqoop-to-hive" xmlns="uri:oozie:workflow:0.4"> > > <start to="sqoop2hive"/> > > <action name="sqoop2hive"> > > <sqoop xmlns="uri:oozie:sqoop-action:0.2"> > > <job-tracker>${jobTracker}</job-tracker> > > <name-node>${nameNode}</name-node> > > <command>import --connect > "jdbc:teradata://server.co/database=TS"-–driver > Com.teradata.jdbc.TeraDriver--username sqoop --password sqoop --table test > --hive-import --hive- table test</command> > > <archive>tdgssconfig.jar</archive> > > <archive>terajdbc4.jar</archive> > > <file>hive-site.xml</file> > > </sqoop> > > <ok to="end"/> > > <error to="kill"/> > > </action> > > <kill name="kill"> > > <message>Action failed</message> > > </kill> > > <end name="end"/> > > </workflow-app> > > > > Please suggest > > > > Regards, > > Chhaya > > > > > > > > >
