> On June 7, 2013, 8:47 p.m., Rohini Palaniswamy wrote: > > trunk/core/src/main/java/org/apache/oozie/action/hadoop/HiveActionExecutor.java, > > line 62 > > <https://reviews.apache.org/r/11680/diff/1/?file=301395#file301395line62> > > > > Can we can now throw RunTimeException here? Same for other action > > executors like pig. > > Robert Kanter wrote: > That should be fine for a deployment, but during the tests it was causing > many to fail because they load the services, including ActionService, which > then tries to initialize all of the ActionExecutors, which then tries to > create the launcher jar by finding the Main classes, which are not available > to core during testing. > > I agree that it would be better to throw a RuntimeException: I'll look > into this some more; perhaps I can tweak a few tests to make it work. > > Rohini Palaniswamy wrote: > Can't we have those ActionExecutors configured only in their specific > sharelib tests, so that core tests don't fail? Deployment is more important > than tests. So if we can tweak the tests, it would be good. > > Robert Kanter wrote: > To try to fix this, I had originally set oozie.action.use.launcher.jar to > false in hsqldb-oozie-site.xml because that's what XTestCase loads by default > for oozie-site. I didn't realize that some of the tests load their own > special oozie-site file, and then they start the Services; with the > RuntimeExceptions added back in, this causes all kinds of weird failures > (e.g. "too many open files" and some others). > > I just tried having XTestCase set oozie.action.use.launcher.jar to false > regardless of which oozie-site is read in by injecting it into the file, and > then it gets rid of the weird failures and leaves only a handful of classes > that failed because of the actual RuntimeException; I'll see if I can > tweak/refactor these. I'll make XTestCase only inject the property if not > already set in the file, so that someone can set it if they really want to.
I managed to get it to work by deleting some duplicate code from TestMapReduceActionError, moving TestMapReduceActionExecutor to the Streaming sharelib, and fixing a minor bug I introduced in the Test___ActionExecutor classes related to the launcher jar. I'll double check by running all of the tests again to make sure, but I think this is now solved :) - Robert ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/11680/#review21594 ----------------------------------------------------------- On June 7, 2013, 8:21 p.m., Robert Kanter wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/11680/ > ----------------------------------------------------------- > > (Updated June 7, 2013, 8:21 p.m.) > > > Review request for oozie. > > > Description > ------- > > The patch is mostly moving some files from core to the Oozie sharelib. There > was some minor refactoring to fix some issues with tests. Otherwise, the only > actual code changes was to add the option to use the launcher jar or not > (oozie.action.use.launcher.jar). > > > This addresses bug OOZIE-1315. > https://issues.apache.org/jira/browse/OOZIE-1315 > > > Diffs > ----- > > trunk/core/pom.xml 1490336 > trunk/core/src/main/java/org/apache/oozie/action/hadoop/ActionStats.java > 1490336 > trunk/core/src/main/java/org/apache/oozie/action/hadoop/ActionType.java > 1490336 > > trunk/core/src/main/java/org/apache/oozie/action/hadoop/FSLauncherURIHandler.java > 1490336 > > trunk/core/src/main/java/org/apache/oozie/action/hadoop/HCatLauncherURIHandler.java > 1490336 > > trunk/core/src/main/java/org/apache/oozie/action/hadoop/HiveActionExecutor.java > 1490336 > > trunk/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java > 1490336 > > trunk/core/src/main/java/org/apache/oozie/action/hadoop/LauncherException.java > 1490336 > trunk/core/src/main/java/org/apache/oozie/action/hadoop/LauncherMain.java > 1490336 > trunk/core/src/main/java/org/apache/oozie/action/hadoop/LauncherMapper.java > 1490336 > > trunk/core/src/main/java/org/apache/oozie/action/hadoop/LauncherMapperHelper.java > PRE-CREATION > > trunk/core/src/main/java/org/apache/oozie/action/hadoop/LauncherURIHandler.java > 1490336 > > trunk/core/src/main/java/org/apache/oozie/action/hadoop/LauncherURIHandlerFactory.java > 1490336 > trunk/core/src/main/java/org/apache/oozie/action/hadoop/MRStats.java > 1490336 > > trunk/core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java > 1490336 > trunk/core/src/main/java/org/apache/oozie/action/hadoop/MapReduceMain.java > 1490336 > > trunk/core/src/main/java/org/apache/oozie/action/hadoop/PigActionExecutor.java > 1490336 > trunk/core/src/main/java/org/apache/oozie/action/hadoop/PipesMain.java > 1490336 > > trunk/core/src/main/java/org/apache/oozie/action/hadoop/PrepareActionsDriver.java > 1490336 > trunk/core/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java > 1490336 > > trunk/core/src/main/java/org/apache/oozie/action/hadoop/SqoopActionExecutor.java > 1490336 > trunk/core/src/main/java/org/apache/oozie/util/HCatURI.java 1490336 > trunk/core/src/main/resources/oozie-default.xml 1490336 > > trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestFSPrepareActions.java > 1490336 > > trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestHCatPrepareActions.java > 1490336 > > trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java > 1490336 > trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestLauncher.java > 1490336 > > trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionError.java > 1490336 > > trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java > 1490336 > > trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutorUberJar.java > 1490336 > > trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestPrepareActionsDriver.java > 1490336 > > trunk/core/src/test/java/org/apache/oozie/action/hadoop/TestShellActionExecutor.java > 1490336 > > trunk/core/src/test/java/org/apache/oozie/command/wf/TestActionCheckXCommand.java > 1490336 > > trunk/core/src/test/java/org/apache/oozie/command/wf/TestActionStartXCommand.java > 1490336 > trunk/core/src/test/java/org/apache/oozie/service/TestRecoveryService.java > 1490336 > trunk/core/src/test/java/org/apache/oozie/util/TestHCatURI.java 1490336 > trunk/core/src/test/resources/hsqldb-oozie-site.xml 1490336 > trunk/examples/pom.xml 1490336 > trunk/examples/src/test/resources/hsqldb-oozie-site.xml 1490336 > trunk/minitest/src/test/resources/hsqldb-oozie-site.xml 1490336 > trunk/pom.xml 1490336 > trunk/sharelib/distcp/pom.xml 1490336 > trunk/sharelib/hcatalog/pom.xml 1490336 > trunk/sharelib/hive/pom.xml 1490336 > > trunk/sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveActionExecutor.java > 1490336 > trunk/sharelib/oozie/pom.xml 1490336 > > trunk/sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMapper.java > PRE-CREATION > > trunk/sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/MapReduceMain.java > PRE-CREATION > trunk/sharelib/pig/pom.xml 1490336 > > trunk/sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigActionExecutor.java > 1490336 > trunk/sharelib/sqoop/pom.xml 1490336 > > trunk/sharelib/sqoop/src/test/java/org/apache/oozie/action/hadoop/TestSqoopActionExecutor.java > 1490336 > trunk/sharelib/streaming/pom.xml 1490336 > trunk/tools/src/test/resources/hsqldb-oozie-site.xml 1490336 > trunk/webapp/pom.xml 1490336 > > Diff: https://reviews.apache.org/r/11680/diff/ > > > Testing > ------- > > - Unit tests for oozie.action.use.launcher.jar > - I verified that it works correctly with oozie.action.use.launcher.jar set > to true and false. > > > Thanks, > > Robert Kanter > >
