> On Oct. 29, 2013, 8:11 p.m., Virag Kothari wrote: > > http://svn.apache.org/repos/asf/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java, > > line 478 > > <https://reviews.apache.org/r/14741/diff/5/?file=372236#file372236line478> > > > > if listOfpaths is null, then we need to have previous logic of reading > > the files from hdfs and adding to cache.
Sharelib is empty, trying to repopulate them. - Purshotam ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/14741/#review27715 ----------------------------------------------------------- On Nov. 1, 2013, 5:58 p.m., Purshotam Shah wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/14741/ > ----------------------------------------------------------- > > (Updated Nov. 1, 2013, 5:58 p.m.) > > > Review request for oozie. > > > Bugs: OOZIE-1584 > https://issues.apache.org/jira/browse/OOZIE-1584 > > > Repository: oozie > > > Description > ------- > > Supported features. > ----------------- > 1. If Oozie.action.ship.launcher.jar = true then, Oozie auto-deploy jar at > server start. > > 2. Admin copies sharelib to new timestamp directory and issues invalidate > command to Oozie server. Oozie picks latest shared lib. > > 3. Oozie server doesn't auto deploy sharelib at startup. It picks the latest > sharelib jar pushed by admin(based on timestamp). > > 4. Sharelib mapping files. > Sharelib mapping file can be also configured.Configured file is a key value > mapping, where key will be the action and value is DFS directory of sharelib > files for action. > This can be configured in oozie-site.xml as : > <property> > <name>oozie.service.ShareLibService.mapping.file</name> > <value></value> > <description> > Sharelib mapping files contains list of key=value, > where key is the action key and value is the DFS location of > sharelib files. > </description> > </property> > > -------------------- > DFS after generating launcher jar by ooozie server and sharelib by cli > > makebag-lm:example purushah$ /var/hadoop-1.2.1/bin/hadoop fs -ls > /user/purushah/share/lib/ > /user/purushah/share/lib/launcher_20131017092254 > /user/purushah/share/lib/launcher_20131017093814 > /user/purushah/share/lib/launcher_20131017094652 > /user/purushah/share/lib/launcher_20131017094836 > /user/purushah/share/lib/launcher_20131017095549 > /user/purushah/share/lib/lib_20131017092806 > makebag-lm:example purushah$ > > > Purging. > ------------- > There are two set( launcher_ for launcher jars and Lib_ for shared lib) of > directory and purging happens for both at startup. > Anything older than ( defined in property) is purged. > We always keep 2 set of directory, purging happens if number of sharelib > directory > 2. > > > Multiple version. > ------------------- > To support multiple version, sharelib will be in below format. > > Sharelib will cache all versions and return list of dfs files for a particular > version. > > makebag-lm:example purushah$ /var/hadoop-1.2.1/bin/hadoop fs -ls > /user/purushah/share/lib/ > Found 15 items > /user/purushah/share/lib/launcher_20131017092254 > /user/purushah/share/lib/launcher_20131017093814 > /user/purushah/share/lib/launcher_20131017094652 > /user/purushah/share/lib/launcher_20131017094836 > /user/purushah/share/lib/launcher_20131017095549 > /user/purushah/share/lib/lib_20131017092806/pig/... > /user/purushah/share/lib/lib_20131017092806/pig_9/... > /user/purushah/share/lib/lib_20131017092806/pig_10/... > makebag-lm:example purushah$ > > > Diffs > ----- > > > http://svn.apache.org/repos/asf/oozie/trunk/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java > 1532127 > > http://svn.apache.org/repos/asf/oozie/trunk/core/src/main/java/org/apache/oozie/service/ShareLibService.java > 1532127 > > http://svn.apache.org/repos/asf/oozie/trunk/core/src/main/resources/oozie-default.xml > 1532127 > > http://svn.apache.org/repos/asf/oozie/trunk/core/src/test/java/org/apache/oozie/service/TestShareLibService.java > 1532127 > > http://svn.apache.org/repos/asf/oozie/trunk/docs/src/site/twiki/AG_Install.twiki > 1532127 > > http://svn.apache.org/repos/asf/oozie/trunk/docs/src/site/twiki/DG_QuickStart.twiki > 1532127 > > http://svn.apache.org/repos/asf/oozie/trunk/tools/src/main/java/org/apache/oozie/tools/OozieSharelibCLI.java > 1532127 > > Diff: https://reviews.apache.org/r/14741/diff/ > > > Testing > ------- > > > Thanks, > > Purshotam Shah > >
