----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/14741/#review28053 -----------------------------------------------------------
http://svn.apache.org/repos/asf/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java <https://reviews.apache.org/r/14741/#comment54913> Move creation of fs after this based on systemLibPath and move it out of for loop http://svn.apache.org/repos/asf/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java <https://reviews.apache.org/r/14741/#comment54573> Use a different method name like getActionShareLibJars http://svn.apache.org/repos/asf/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java <https://reviews.apache.org/r/14741/#comment54909> Rename to addShippedLauncherShareLibForAction http://svn.apache.org/repos/asf/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java <https://reviews.apache.org/r/14741/#comment54908> Revert /// http://svn.apache.org/repos/asf/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java <https://reviews.apache.org/r/14741/#comment54948> Call getActionLauncherJars. return if listOfPaths is null http://svn.apache.org/repos/asf/oozie/trunk/core/src/main/java/org/apache/oozie/service/ShareLibService.java <https://reviews.apache.org/r/14741/#comment54932> rename to launcherJarsMap and have a separate shareLibMap http://svn.apache.org/repos/asf/oozie/trunk/core/src/main/java/org/apache/oozie/service/ShareLibService.java <https://reviews.apache.org/r/14741/#comment54850> Construct the tmp path inside the if block. http://svn.apache.org/repos/asf/oozie/trunk/core/src/main/java/org/apache/oozie/service/ShareLibService.java <https://reviews.apache.org/r/14741/#comment54933> Extract this out to a separate method shipLauncherJars http://svn.apache.org/repos/asf/oozie/trunk/core/src/main/java/org/apache/oozie/service/ShareLibService.java <https://reviews.apache.org/r/14741/#comment54848> Make this as a local variable instead of parsing every time http://svn.apache.org/repos/asf/oozie/trunk/core/src/main/java/org/apache/oozie/service/ShareLibService.java <https://reviews.apache.org/r/14741/#comment54849> It does not need launcherlibPath. Needs the system lib path only. Also log a error message if exception in purge. don't fail http://svn.apache.org/repos/asf/oozie/trunk/core/src/main/java/org/apache/oozie/service/ShareLibService.java <https://reviews.apache.org/r/14741/#comment54915> Should not be needing this on init. http://svn.apache.org/repos/asf/oozie/trunk/core/src/main/java/org/apache/oozie/service/ShareLibService.java <https://reviews.apache.org/r/14741/#comment54925> Combine into one log statement. http://svn.apache.org/repos/asf/oozie/trunk/core/src/main/java/org/apache/oozie/service/ShareLibService.java <https://reviews.apache.org/r/14741/#comment54917> launcherLibPath http://svn.apache.org/repos/asf/oozie/trunk/core/src/main/java/org/apache/oozie/service/ShareLibService.java <https://reviews.apache.org/r/14741/#comment54874> This code should not be removed http://svn.apache.org/repos/asf/oozie/trunk/core/src/main/java/org/apache/oozie/service/ShareLibService.java <https://reviews.apache.org/r/14741/#comment54919> Not required http://svn.apache.org/repos/asf/oozie/trunk/core/src/main/java/org/apache/oozie/service/ShareLibService.java <https://reviews.apache.org/r/14741/#comment54927> Rename method to getActionLauncherJars http://svn.apache.org/repos/asf/oozie/trunk/core/src/main/java/org/apache/oozie/service/ShareLibService.java <https://reviews.apache.org/r/14741/#comment54947> if shipLauncher setting is true && http://svn.apache.org/repos/asf/oozie/trunk/core/src/main/java/org/apache/oozie/service/ShareLibService.java <https://reviews.apache.org/r/14741/#comment54949> copyLauncherJars http://svn.apache.org/repos/asf/oozie/trunk/core/src/main/java/org/apache/oozie/service/ShareLibService.java <https://reviews.apache.org/r/14741/#comment54946> throw the exception. do not log. http://svn.apache.org/repos/asf/oozie/trunk/core/src/main/java/org/apache/oozie/service/ShareLibService.java <https://reviews.apache.org/r/14741/#comment54929> inline this method in init() and remove this method. http://svn.apache.org/repos/asf/oozie/trunk/core/src/main/java/org/apache/oozie/service/ShareLibService.java <https://reviews.apache.org/r/14741/#comment54922> Do purge irrespective of the setting. http://svn.apache.org/repos/asf/oozie/trunk/core/src/main/java/org/apache/oozie/service/ShareLibService.java <https://reviews.apache.org/r/14741/#comment54923> Do purge irrespective of the setting. http://svn.apache.org/repos/asf/oozie/trunk/core/src/main/java/org/apache/oozie/service/ShareLibService.java <https://reviews.apache.org/r/14741/#comment54924> launcherJarLibPath is not required in method signature http://svn.apache.org/repos/asf/oozie/trunk/core/src/main/java/org/apache/oozie/service/ShareLibService.java <https://reviews.apache.org/r/14741/#comment54875> how does this work without prefix? http://svn.apache.org/repos/asf/oozie/trunk/core/src/main/java/org/apache/oozie/service/ShareLibService.java <https://reviews.apache.org/r/14741/#comment54934> Method can be removed http://svn.apache.org/repos/asf/oozie/trunk/core/src/main/java/org/apache/oozie/service/ShareLibService.java <https://reviews.apache.org/r/14741/#comment54935> Call method as cacheShareLibMapping http://svn.apache.org/repos/asf/oozie/trunk/core/src/main/java/org/apache/oozie/service/ShareLibService.java <https://reviews.apache.org/r/14741/#comment54936> Use a different map for this - shareLibMap http://svn.apache.org/repos/asf/oozie/trunk/core/src/main/java/org/apache/oozie/service/ShareLibService.java <https://reviews.apache.org/r/14741/#comment54938> Why is type required in method signature. Can be removed http://svn.apache.org/repos/asf/oozie/trunk/core/src/main/java/org/apache/oozie/service/ShareLibService.java <https://reviews.apache.org/r/14741/#comment54944> properties http://svn.apache.org/repos/asf/oozie/trunk/core/src/main/java/org/apache/oozie/service/ShareLibService.java <https://reviews.apache.org/r/14741/#comment54943> PropertiesFile http://svn.apache.org/repos/asf/oozie/trunk/core/src/main/java/org/apache/oozie/service/ShareLibService.java <https://reviews.apache.org/r/14741/#comment54942> Can remove this method http://svn.apache.org/repos/asf/oozie/trunk/core/src/main/java/org/apache/oozie/service/ShareLibService.java <https://reviews.apache.org/r/14741/#comment54940> LauncherLibPath http://svn.apache.org/repos/asf/oozie/trunk/core/src/main/java/org/apache/oozie/service/ShareLibService.java <https://reviews.apache.org/r/14741/#comment54941> tmpLauncherLibPath http://svn.apache.org/repos/asf/oozie/trunk/core/src/main/java/org/apache/oozie/service/ShareLibService.java <https://reviews.apache.org/r/14741/#comment54937> getLatestLibPath http://svn.apache.org/repos/asf/oozie/trunk/core/src/main/java/org/apache/oozie/service/ShareLibService.java <https://reviews.apache.org/r/14741/#comment54876> have this as a local variable http://svn.apache.org/repos/asf/oozie/trunk/docs/src/site/twiki/DG_QuickStart.twiki <https://reviews.apache.org/r/14741/#comment54877> Remove upgrade http://svn.apache.org/repos/asf/oozie/trunk/docs/src/site/twiki/DG_QuickStart.twiki <https://reviews.apache.org/r/14741/#comment54878> remove "new or upgrading existing" http://svn.apache.org/repos/asf/oozie/trunk/docs/src/site/twiki/DG_QuickStart.twiki <https://reviews.apache.org/r/14741/#comment54879> Specify the format lib_<timestamp> timestamped instead of time-stamp - Rohini Palaniswamy On Nov. 1, 2013, 5:58 p.m., Purshotam Shah wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/14741/ > ----------------------------------------------------------- > > (Updated Nov. 1, 2013, 5:58 p.m.) > > > Review request for oozie. > > > Bugs: OOZIE-1584 > https://issues.apache.org/jira/browse/OOZIE-1584 > > > Repository: oozie > > > Description > ------- > > Supported features. > ----------------- > 1. If Oozie.action.ship.launcher.jar = true then, Oozie auto-deploy jar at > server start. > > 2. Admin copies sharelib to new timestamp directory and issues invalidate > command to Oozie server. Oozie picks latest shared lib. > > 3. Oozie server doesn't auto deploy sharelib at startup. It picks the latest > sharelib jar pushed by admin(based on timestamp). > > 4. Sharelib mapping files. > Sharelib mapping file can be also configured.Configured file is a key value > mapping, where key will be the action and value is DFS directory of sharelib > files for action. > This can be configured in oozie-site.xml as : > <property> > <name>oozie.service.ShareLibService.mapping.file</name> > <value></value> > <description> > Sharelib mapping files contains list of key=value, > where key is the action key and value is the DFS location of > sharelib files. > </description> > </property> > > -------------------- > DFS after generating launcher jar by ooozie server and sharelib by cli > > makebag-lm:example purushah$ /var/hadoop-1.2.1/bin/hadoop fs -ls > /user/purushah/share/lib/ > /user/purushah/share/lib/launcher_20131017092254 > /user/purushah/share/lib/launcher_20131017093814 > /user/purushah/share/lib/launcher_20131017094652 > /user/purushah/share/lib/launcher_20131017094836 > /user/purushah/share/lib/launcher_20131017095549 > /user/purushah/share/lib/lib_20131017092806 > makebag-lm:example purushah$ > > > Purging. > ------------- > There are two set( launcher_ for launcher jars and Lib_ for shared lib) of > directory and purging happens for both at startup. > Anything older than ( defined in property) is purged. > We always keep 2 set of directory, purging happens if number of sharelib > directory > 2. > > > Multiple version. > ------------------- > To support multiple version, sharelib will be in below format. > > Sharelib will cache all versions and return list of dfs files for a particular > version. > > makebag-lm:example purushah$ /var/hadoop-1.2.1/bin/hadoop fs -ls > /user/purushah/share/lib/ > Found 15 items > /user/purushah/share/lib/launcher_20131017092254 > /user/purushah/share/lib/launcher_20131017093814 > /user/purushah/share/lib/launcher_20131017094652 > /user/purushah/share/lib/launcher_20131017094836 > /user/purushah/share/lib/launcher_20131017095549 > /user/purushah/share/lib/lib_20131017092806/pig/... > /user/purushah/share/lib/lib_20131017092806/pig_9/... > /user/purushah/share/lib/lib_20131017092806/pig_10/... > makebag-lm:example purushah$ > > > Diffs > ----- > > > http://svn.apache.org/repos/asf/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java > 1532127 > > http://svn.apache.org/repos/asf/oozie/trunk/core/src/main/java/org/apache/oozie/service/ShareLibService.java > 1532127 > > http://svn.apache.org/repos/asf/oozie/trunk/core/src/main/resources/oozie-default.xml > 1532127 > > http://svn.apache.org/repos/asf/oozie/trunk/core/src/test/java/org/apache/oozie/service/TestShareLibService.java > 1532127 > > http://svn.apache.org/repos/asf/oozie/trunk/docs/src/site/twiki/AG_Install.twiki > 1532127 > > http://svn.apache.org/repos/asf/oozie/trunk/docs/src/site/twiki/DG_QuickStart.twiki > 1532127 > > http://svn.apache.org/repos/asf/oozie/trunk/tools/src/main/java/org/apache/oozie/tools/OozieSharelibCLI.java > 1532127 > > Diff: https://reviews.apache.org/r/14741/diff/ > > > Testing > ------- > > > Thanks, > > Purshotam Shah > >
