Hey Ron, I think that's a cool idea. Using Sqoop as a library makes a lot of sense to me. Sqoop2 was supposed to solve a lot of these concerns, but feel free to post something on dev@ about your idea.
-Abe On Wed, Jul 29, 2015 at 5:40 PM, Ron Gonzalez <[email protected]> wrote: > I've refactored Sqoop such that I am able to invoke it asynchronously > such that I can use Sqoop to just submit the job then return the > application id, and I've wrapped it behind a REST API that returns the > application id. I didn't have to use Sqoop2 or anything like that. > > I'm also wondering if this is something that I could contribute back to > the Sqoop code base as an option that allows Sqoop to be used as a library > instead of just as a command line tool. > > Thanks, > Ron > > > On 07/29/2015 05:14 PM, Abraham Elmahrek wrote: > > This isn't something usually done (shell script calling Sqoop). The shell > action can call the Sqoop command with an absolute path, but you'll run > into classpath issues. I haven't personally trouble shot this, but it > should work if you resolve all the classpath issues. > > Check out http://sqoop.apache.org/docs/1.99.6/index.html for information > on Sqoop2. > > -Abe > > On Wed, Jul 29, 2015 at 5:06 PM, <[email protected]> wrote: > >> On a second thought, I was planning to execute it with shell script. >> >> So, my shell script has a sqoop command. >> >> SQOOP is installed on all the nodes. >> >> >> >> But, when I run with OOZIE, I get a frustratingly small error. >> >> >> >> Log Type: stderr >> >> Log Length: 362 >> >> ./abcd.sh: line 3: sqoop: command not found >> >> Failing Oozie Launcher, Main class >> [org.apache.oozie.action.hadoop.ShellMain], exit code [1] >> >> log4j:WARN No appenders could be found for logger >> (amazon.emr.metrics.MetricsSaver). >> >> log4j:WARN Please initialize the log4j system properly. >> >> log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for >> more info. >> >> >> >> Log Type: stdout >> >> >> >> Any idea, folks ? >> >> >> >> >> >> >> >> *From:* Abraham Elmahrek [mailto:[email protected]] >> *Sent:* Wednesday, July 29, 2015 4:17 PM >> *To:* [email protected] >> *Subject:* Re: Anyone ran a sqoop job using JAVA code >> >> >> >> Have you tried Sqoop2 for Java integration? >> >> >> >> -Abe >> >> >> >> On Wed, Jul 29, 2015 at 3:28 PM, <[email protected]> wrote: >> >> Hi all, >> >> Is there anyone who has written JAVA code in which “SQOOP job –exec >> JOBID” is being carried out? >> >> >> >> Help is appreciated. >> >> >> >> Regards, >> >> >> >> *Shouvanik Haldar* >> >> >> ------------------------------ >> >> >> This message is for the designated recipient only and may contain >> privileged, proprietary, or otherwise confidential information. If you have >> received it in error, please notify the sender immediately and delete the >> original. Any other use of the e-mail by you is prohibited. Where allowed >> by local law, electronic communications with Accenture and its affiliates, >> including e-mail and instant messaging (including content), may be scanned >> by our systems for the purposes of information security and assessment of >> internal compliance with Accenture policy. >> >> ______________________________________________________________________________________ >> >> www.accenture.com >> >> >> > > >
