I've refactored Sqoop such that I am able to invoke it asynchronously
such that I can use Sqoop to just submit the job then return the
application id, and I've wrapped it behind a REST API that returns the
application id. I didn't have to use Sqoop2 or anything like that.
I'm also wondering if this is something that I could contribute back to
the Sqoop code base as an option that allows Sqoop to be used as a
library instead of just as a command line tool.
Thanks,
Ron
On 07/29/2015 05:14 PM, Abraham Elmahrek wrote:
This isn't something usually done (shell script calling Sqoop). The
shell action can call the Sqoop command with an absolute path, but
you'll run into classpath issues. I haven't personally trouble shot
this, but it should work if you resolve all the classpath issues.
Check out http://sqoop.apache.org/docs/1.99.6/index.html for
information on Sqoop2.
-Abe
On Wed, Jul 29, 2015 at 5:06 PM, <[email protected]
<mailto:[email protected]>> wrote:
On a second thought, I was planning to execute it with shell script.
So, my shell script has a sqoop command.
SQOOP is installed on all the nodes.
But, when I run with OOZIE, I get a frustratingly small error.
Log Type: stderr
Log Length: 362
./abcd.sh: line 3: sqoop: command not found
Failing Oozie Launcher, Main class
[org.apache.oozie.action.hadoop.ShellMain], exit code [1]
log4j:WARN No appenders could be found for logger
(amazon.emr.metrics.MetricsSaver).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See
http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
Log Type: stdout
Any idea, folks ?
*From:*Abraham Elmahrek [mailto:[email protected]
<mailto:[email protected]>]
*Sent:* Wednesday, July 29, 2015 4:17 PM
*To:* [email protected] <mailto:[email protected]>
*Subject:* Re: Anyone ran a sqoop job using JAVA code
Have you tried Sqoop2 for Java integration?
-Abe
On Wed, Jul 29, 2015 at 3:28 PM, <[email protected]
<mailto:[email protected]>> wrote:
Hi all,
Is there anyone who has written JAVA code in which “SQOOP job
–exec JOBID” is being carried out?
Help is appreciated.
Regards,
*Shouvanik Haldar*
------------------------------------------------------------------------
This message is for the designated recipient only and may
contain privileged, proprietary, or otherwise confidential
information. If you have received it in error, please notify
the sender immediately and delete the original. Any other use
of the e-mail by you is prohibited. Where allowed by local
law, electronic communications with Accenture and its
affiliates, including e-mail and instant messaging (including
content), may be scanned by our systems for the purposes of
information security and assessment of internal compliance
with Accenture policy.
______________________________________________________________________________________
www.accenture.com <http://www.accenture.com>
