[
https://issues.apache.org/jira/browse/OOZIE-1521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shuaishuai Nie updated OOZIE-1521:
----------------------------------
Attachment: OOZIE-1521.1.patch
> Optimize I/O for Sqoop workflow launched by Oozie
> -------------------------------------------------
>
> Key: OOZIE-1521
> URL: https://issues.apache.org/jira/browse/OOZIE-1521
> Project: Oozie
> Issue Type: Improvement
> Reporter: Shuaishuai Nie
> Attachments: OOZIE-1521.1.patch
>
>
> Now Sqoop work flow launched by Oozie will copy the jar files in
> %SQOOP_HOME%\lib folder to the job cache each time a Sqoop job is launched.
> We should add these jars to Sqoop sharelib so that for each worker node, the
> jars in the Sqoop sharelib only copied to distribute cache and localize to
> each worker node once and reuse by all the subsequent Sqoop jobs. In
> Sqoop-1192, an option 'skip-dist-cache' is introduced which disable Sqoop add
> lib jars to job cache action and reuse the jars in the Sqoop sharelib. In the
> patch attached, Oozie will add "--skip-dist-cache" option to Sqoop argument
> if share lib is used. Also add the Sqoop dependencies in the Sqoop share lib.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
