[jira] Subscription: Oozie Patch Available
Issue Subscription Filter: Oozie Patch Available (109 issues) Subscriber: ooziedaily Key Summary OOZIE-3082 Move org.apache.oozie.action.hadoop.ActionStats to Oozie ShareLib https://issues.apache.org/jira/browse/OOZIE-3082 OOZIE-3081 testSubmitWithLauncherQueue fails when FairScheduler is used https://issues.apache.org/jira/browse/OOZIE-3081 OOZIE-3071 Oozie 4.3 Spark sharelib ueses a different version of commons-lang3 than Spark 2.2.0 https://issues.apache.org/jira/browse/OOZIE-3071 OOZIE-3063 Sanitizing variables that are part of openjpa.ConnectionProperties https://issues.apache.org/jira/browse/OOZIE-3063 OOZIE-3062 Set HADOOP_CONF_DIR for spark action https://issues.apache.org/jira/browse/OOZIE-3062 OOZIE-3022 fix for warning has no file and won't be listed in dependency files details https://issues.apache.org/jira/browse/OOZIE-3022 OOZIE-3002 address findbugs errors in client lib https://issues.apache.org/jira/browse/OOZIE-3002 OOZIE-3001 core library has many instances of warnings with trailing spaces and lines longer than 132 chars https://issues.apache.org/jira/browse/OOZIE-3001 OOZIE-2997 files contain trailing white spaces in client lib https://issues.apache.org/jira/browse/OOZIE-2997 OOZIE-2996 add option for -UseGCOverheadLimit to maven opts as sometimes local testing fails https://issues.apache.org/jira/browse/OOZIE-2996 OOZIE-2978 Remove code that handles Pig versions before 0.8 https://issues.apache.org/jira/browse/OOZIE-2978 OOZIE-2975 code clean up in pig sharelib, replace Exception with more explicit, add try with resources, StringBuilder instead of StringBuffer https://issues.apache.org/jira/browse/OOZIE-2975 OOZIE-2969 Drop support for Java 1.7 https://issues.apache.org/jira/browse/OOZIE-2969 OOZIE-2964 Add -Xdoclint:none to javadoc opts to avoid warnings https://issues.apache.org/jira/browse/OOZIE-2964 OOZIE-2962 bump maven-javadoc-plugin to 2.10.4 https://issues.apache.org/jira/browse/OOZIE-2962 OOZIE-2957 Documentation states that starting a coordinator is possible https://issues.apache.org/jira/browse/OOZIE-2957 OOZIE-2956 Fix Findbugs warnings related to reliance on default encoding in oozie-core https://issues.apache.org/jira/browse/OOZIE-2956 OOZIE-2955 Fix Findbugs warnings related to reliance on default encoding in oozie-client https://issues.apache.org/jira/browse/OOZIE-2955 OOZIE-2954 Fix Checkstyle issues in oozie-client https://issues.apache.org/jira/browse/OOZIE-2954 OOZIE-2953 Fix Checkstyle issues in oozie-tools https://issues.apache.org/jira/browse/OOZIE-2953 OOZIE-2952 Fix Findbugs warnings in oozie-sharelib-oozie https://issues.apache.org/jira/browse/OOZIE-2952 OOZIE-2949 Escape quotes whitespaces in Sqoop field https://issues.apache.org/jira/browse/OOZIE-2949 OOZIE-2942 Fix Findbugs warnings in oozie-examples https://issues.apache.org/jira/browse/OOZIE-2942 OOZIE-2937 Remove redundant groupId from the child pom's https://issues.apache.org/jira/browse/OOZIE-2937 OOZIE-2934 Fix "Exceptional return value of java.io.File.mkdirs() ignored" Findbugs error in oozie-sharelib-spark https://issues.apache.org/jira/browse/OOZIE-2934 OOZIE-2927 Append new line character for Hive2 query using query tag https://issues.apache.org/jira/browse/OOZIE-2927 OOZIE-2914 Consolidate Trim https://issues.apache.org/jira/browse/OOZIE-2914 OOZIE-2896 Ensure compatibility for existing LauncherMapper settings https://issues.apache.org/jira/browse/OOZIE-2896 OOZIE-2883 OOZIE throw the error "Missing [oozie.service.ProxyUserService.proxyuser.oozie.service.ProxyUserService.proxyuser.mr.groups] property" https://issues.apache.org/jira/browse/OOZIE-2883 OOZIE-2877 Oozie Git Action https://issues.apache.org/jira/browse/OOZIE-2877 OOZIE-2867 Timezone handling for Coordinators: emphasize "Continent/City" format https://issues.apache.org/jira/browse/OOZIE-2867 OOZIE-2834 ParameterVerifier logging non-useful warning for workflow definition https://issues.apache.org/jira/browse/OOZIE-2834 OOZIE-2833 when using uber mode the regex pattern used in the extractHeapSizeMB method does not allow heap sizes specified in bytes. https://issues.apache.org/jira/browse/OOZIE-2833 OOZIE-2829 Improve sharelib upload to accept multiple source folders https://issues.apache.org/jira/browse/OOZIE-2829 OOZIE-2826 Falcon feed fails to aws s3; Oozie joda time version does not meet required jar version 2.2 or later https://issues.apache.org/jira/browse/OOZIE-2826 OOZIE-2812 SparkConfigurationService should support loading configurations from multiple Spark
Failed: OOZIE-2896 PreCommit Build #98
Jira: https://issues.apache.org/jira/browse/OOZIE-2896 Build: https://builds.apache.org/job/PreCommit-OOZIE-Build/98/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated 1.53 MB...] -1 the patch contains 1 line(s) longer than 132 characters +1 the patch does adds/modifies 2 testcase(s) +1 RAT +1 the patch does not seem to introduce new RAT warnings +1 JAVADOC +1 the patch does not seem to introduce new Javadoc warnings WARNING: the current HEAD has 77 Javadoc warning(s) +1 COMPILE +1 HEAD compiles +1 patch compiles +1 the patch does not seem to introduce new javac warnings +1 There are no new bugs found in total. +1 There are no new bugs found in [client]. +1 There are no new bugs found in [sharelib/hive]. +1 There are no new bugs found in [sharelib/sqoop]. +1 There are no new bugs found in [sharelib/spark]. +1 There are no new bugs found in [sharelib/streaming]. +1 There are no new bugs found in [sharelib/hive2]. +1 There are no new bugs found in [sharelib/oozie]. +1 There are no new bugs found in [sharelib/hcatalog]. +1 There are no new bugs found in [sharelib/pig]. +1 There are no new bugs found in [sharelib/distcp]. +1 There are no new bugs found in [docs]. +1 There are no new bugs found in [server]. +1 There are no new bugs found in [core]. +1 There are no new bugs found in [examples]. +1 There are no new bugs found in [tools]. +1 BACKWARDS_COMPATIBILITY +1 the patch does not change any JPA Entity/Colum/Basic/Lob/Transient annotations +1 the patch does not modify JPA files -1 TESTS - patch does not compile, cannot run testcases +1 DISTRO +1 distro tarball builds with the patch -1 Overall result, please check the reported -1(s) There is at least one warning, please check The full output of the test-patch run is available at https://builds.apache.org/job/PreCommit-OOZIE-Build/98/ % Total% Received % Xferd Average Speed TimeTime Time Current Dload Upload Total SpentLeft Speed 0 00 00 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 00 00 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 3706k 100 3706k0 0 4647k 0 --:--:-- --:--:-- --:--:-- 4647k Adding comment to JIRA Comment added. test-patch exit code: 1 Build step 'Execute shell' marked build as failure [description-setter] Description set: OOZIE-2896 Archiving artifacts [Fast Archiver] Compressed 1.80 MB of artifacts by 45.2% relative to #95 Recording test results Email was triggered for: Failure - Any Sending email for trigger: Failure - Any ### ## FAILED TESTS (if any) ## All tests passed
[jira] [Commented] (OOZIE-2896) Ensure compatibility for existing LauncherMapper settings
[
https://issues.apache.org/jira/browse/OOZIE-2896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16199157#comment-16199157
]
Hadoop QA commented on OOZIE-2896:
--
Testing JIRA OOZIE-2896
Cleaning local git workspace
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:red}-1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:red}-1{color} the patch contains 1 line(s) longer than 132
characters
.{color:green}+1{color} the patch does adds/modifies 2 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
warnings
.{color:red}WARNING{color}: the current HEAD has 77 Javadoc warning(s)
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac
warnings
{color:green}+1{color} There are no new bugs found in total.
. {color:green}+1{color} There are no new bugs found in [client].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive].
. {color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
. {color:green}+1{color} There are no new bugs found in [sharelib/spark].
. {color:green}+1{color} There are no new bugs found in [sharelib/streaming].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive2].
. {color:green}+1{color} There are no new bugs found in [sharelib/oozie].
. {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
. {color:green}+1{color} There are no new bugs found in [sharelib/pig].
. {color:green}+1{color} There are no new bugs found in [sharelib/distcp].
. {color:green}+1{color} There are no new bugs found in [docs].
. {color:green}+1{color} There are no new bugs found in [server].
. {color:green}+1{color} There are no new bugs found in [core].
. {color:green}+1{color} There are no new bugs found in [examples].
. {color:green}+1{color} There are no new bugs found in [tools].
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:red}-1 TESTS{color} - patch does not compile, cannot run testcases
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch
{color:red}*-1 Overall result, please check the reported -1(s)*{color}
{color:red}. There is at least one warning, please check{color}
The full output of the test-patch run is available at
. https://builds.apache.org/job/PreCommit-OOZIE-Build/98/
> Ensure compatibility for existing LauncherMapper settings
> -
>
> Key: OOZIE-2896
> URL: https://issues.apache.org/jira/browse/OOZIE-2896
> Project: Oozie
> Issue Type: Sub-task
>Affects Versions: 5.0.0
>Reporter: Peter Bacsko
>Assignee: Andras Piros
>Priority: Blocker
> Fix For: 5.0.0
>
> Attachments: OOZIE-2896.001.patch
>
>
> There are a couple of LauncherMapper settings that we should support for
> backward compatibility reasons.
> For example, {{oozie.launcher.mapreduce.map.memory.mb}},
> {{mapred.child.env}}, {{mapred.map.env}}, etc.
> We also have to discuss the list of properties we're going to support, since
> there are many of them.
> List of hadoop properties under this task:
> mapreduce.job.queuename
> mapreduce.map.memory.mb
> mapreduce.map.java.opts
> mapreduce.map.env
> mapreduce.job.priority
> yarn.app.mapreduce.am.scheduler.heartbeat.interval-ms
> yarn.app.mapreduce.am.resource.mb
> yarn.app.mapreduce.am.command-opts
> yarn.app.mapreduce.am.env
> ACL related properties should go under OOZIE-2897:
> mapreduce.job.acl-view-job
> mapreduce.job.acl-modfy-job
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
Failed: OOZIE-3081 PreCommit Build #97
Jira: https://issues.apache.org/jira/browse/OOZIE-3081 Build: https://builds.apache.org/job/PreCommit-OOZIE-Build/97/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated 1.52 MB...] +1 the patch does not introduce any line longer than 132 +1 the patch does adds/modifies 1 testcase(s) +1 RAT +1 the patch does not seem to introduce new RAT warnings +1 JAVADOC +1 the patch does not seem to introduce new Javadoc warnings WARNING: the current HEAD has 77 Javadoc warning(s) +1 COMPILE +1 HEAD compiles +1 patch compiles +1 the patch does not seem to introduce new javac warnings +1 There are no new bugs found in total. +1 There are no new bugs found in [docs]. +1 There are no new bugs found in [examples]. +1 There are no new bugs found in [tools]. +1 There are no new bugs found in [core]. +1 There are no new bugs found in [server]. +1 There are no new bugs found in [client]. +1 There are no new bugs found in [sharelib/distcp]. +1 There are no new bugs found in [sharelib/oozie]. +1 There are no new bugs found in [sharelib/hcatalog]. +1 There are no new bugs found in [sharelib/streaming]. +1 There are no new bugs found in [sharelib/hive]. +1 There are no new bugs found in [sharelib/sqoop]. +1 There are no new bugs found in [sharelib/spark]. +1 There are no new bugs found in [sharelib/hive2]. +1 There are no new bugs found in [sharelib/pig]. +1 BACKWARDS_COMPATIBILITY +1 the patch does not change any JPA Entity/Colum/Basic/Lob/Transient annotations +1 the patch does not modify JPA files -1 TESTS - patch does not compile, cannot run testcases +1 DISTRO +1 distro tarball builds with the patch -1 Overall result, please check the reported -1(s) There is at least one warning, please check The full output of the test-patch run is available at https://builds.apache.org/job/PreCommit-OOZIE-Build/97/ % Total% Received % Xferd Average Speed TimeTime Time Current Dload Upload Total SpentLeft Speed 0 00 00 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 00 00 0 0 0 --:--:-- --:--:-- --:--:-- 0 23 3706k 23 875k0 0 1162k 0 0:00:03 --:--:-- 0:00:03 1162k100 3706k 100 3706k0 0 4591k 0 --:--:-- --:--:-- --:--:-- 51.1M Adding comment to JIRA Comment added. test-patch exit code: 1 Build step 'Execute shell' marked build as failure [description-setter] Description set: OOZIE-3081 Archiving artifacts [Fast Archiver] Compressed 1.79 MB of artifacts by 80.2% relative to #95 Recording test results Email was triggered for: Failure - Any Sending email for trigger: Failure - Any ### ## FAILED TESTS (if any) ## All tests passed
[jira] [Commented] (OOZIE-3081) testSubmitWithLauncherQueue fails when FairScheduler is used
[
https://issues.apache.org/jira/browse/OOZIE-3081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16198982#comment-16198982
]
Hadoop QA commented on OOZIE-3081:
--
Testing JIRA OOZIE-3081
Cleaning local git workspace
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:green}+1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than
132
.{color:green}+1{color} the patch does adds/modifies 1 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
warnings
.{color:red}WARNING{color}: the current HEAD has 77 Javadoc warning(s)
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac
warnings
{color:green}+1{color} There are no new bugs found in total.
. {color:green}+1{color} There are no new bugs found in [docs].
. {color:green}+1{color} There are no new bugs found in [examples].
. {color:green}+1{color} There are no new bugs found in [tools].
. {color:green}+1{color} There are no new bugs found in [core].
. {color:green}+1{color} There are no new bugs found in [server].
. {color:green}+1{color} There are no new bugs found in [client].
. {color:green}+1{color} There are no new bugs found in [sharelib/distcp].
. {color:green}+1{color} There are no new bugs found in [sharelib/oozie].
. {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
. {color:green}+1{color} There are no new bugs found in [sharelib/streaming].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive].
. {color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
. {color:green}+1{color} There are no new bugs found in [sharelib/spark].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive2].
. {color:green}+1{color} There are no new bugs found in [sharelib/pig].
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:red}-1 TESTS{color} - patch does not compile, cannot run testcases
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch
{color:red}*-1 Overall result, please check the reported -1(s)*{color}
{color:red}. There is at least one warning, please check{color}
The full output of the test-patch run is available at
. https://builds.apache.org/job/PreCommit-OOZIE-Build/97/
> testSubmitWithLauncherQueue fails when FairScheduler is used
>
>
> Key: OOZIE-3081
> URL: https://issues.apache.org/jira/browse/OOZIE-3081
> Project: Oozie
> Issue Type: Bug
> Components: tests
>Affects Versions: 5.0.0
>Reporter: Attila Sasvari
>Assignee: Attila Sasvari
>Priority: Trivial
> Attachments: OOZIE-3081-001.patch, OOZIE-3081-002.patch
>
>
> Running TestJavaActionExecutor fails with fair scheduler
> {code}
> Failed tests:
> testSubmitWithLauncherQueue(org.apache.oozie.action.hadoop.TestJavaActionExecutor):
>
> expected:<[]test> but was:<[root.]test>
> {code}
> Reason: FairScheduler adds a "root." prefix to the queue name in
> [org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.QueueManager.ensureRootPrefix()|https://github.com/apache/hadoop/blob/99292adcefdc6b8f280b8e100605fb39f755c38a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/QueueManager.java#L165]
> A potential way to fix the test case:
> - we can add a new function
> {code}
> private boolean isFairSchedulerUsed(Configuration conf) {
> return
> conf.get("yarn.resourcemanager.scheduler.class").contains(FairScheduler.class.getName());
> }
> {code}
> that can be called from {{testSubmitWithLauncherQueue()}}:
> {code}
> ...
> if (isFairSchedulerUsed(conf)) {
> assertEquals("root.test", queue);
> }
> else {
> assertEquals("test", queue);
> }
> ...
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (OOZIE-3077) SLA documentation fixes
[ https://issues.apache.org/jira/browse/OOZIE-3077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16198925#comment-16198925 ] Satish Subhashrao Saley commented on OOZIE-3077: Thank you Andras and Rohini for review. Committed to master. > SLA documentation fixes > --- > > Key: OOZIE-3077 > URL: https://issues.apache.org/jira/browse/OOZIE-3077 > Project: Oozie > Issue Type: Bug >Reporter: Satish Subhashrao Saley >Assignee: Satish Subhashrao Saley >Priority: Minor > Attachments: OOZIE-3077-1.patch, OOZIE-3077-2.patch > > > Following properties can be specified in job.xml -> Following properties can > be specified in job.xml, job.properties > Default is 24 hours -> Default is 48 hours > oozie.sla.disable.alerts -> oozie.sla.enable.alerts for property name in the > example configuration having 2009-01-01T01:00Z::2009-05-31T23:59Z as value -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (OOZIE-3083) Make improved version Info backward compatible
[
https://issues.apache.org/jira/browse/OOZIE-3083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Artem Ervits reassigned OOZIE-3083:
---
Assignee: Artem Ervits
> Make improved version Info backward compatible
> --
>
> Key: OOZIE-3083
> URL: https://issues.apache.org/jira/browse/OOZIE-3083
> Project: Oozie
> Issue Type: Bug
>Reporter: Peter Cseh
>Assignee: Artem Ervits
>
> OOZIE-2004 changed the format of the version info output considerably.
> We should change it back to a more compatible format.
> It could be something like
> {code}
> {"buildVersion":"5.0.0-SNAPSHOT", "buildInfo":"{build.time=${build.time},
> vc.url=${vc.url}, build.user=asasvari, vc.revision=${vc.revision}"}
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (OOZIE-2896) Ensure compatibility for existing LauncherMapper settings
[
https://issues.apache.org/jira/browse/OOZIE-2896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andras Piros updated OOZIE-2896:
Attachment: OOZIE-2896.001.patch
> Ensure compatibility for existing LauncherMapper settings
> -
>
> Key: OOZIE-2896
> URL: https://issues.apache.org/jira/browse/OOZIE-2896
> Project: Oozie
> Issue Type: Sub-task
>Affects Versions: 5.0.0
>Reporter: Peter Bacsko
>Assignee: Andras Piros
>Priority: Blocker
> Fix For: 5.0.0
>
> Attachments: OOZIE-2896.001.patch
>
>
> There are a couple of LauncherMapper settings that we should support for
> backward compatibility reasons.
> For example, {{oozie.launcher.mapreduce.map.memory.mb}},
> {{mapred.child.env}}, {{mapred.map.env}}, etc.
> We also have to discuss the list of properties we're going to support, since
> there are many of them.
> List of hadoop properties under this task:
> mapreduce.job.queuename
> mapreduce.map.memory.mb
> mapreduce.map.java.opts
> mapreduce.map.env
> mapreduce.job.priority
> yarn.app.mapreduce.am.scheduler.heartbeat.interval-ms
> yarn.app.mapreduce.am.resource.mb
> yarn.app.mapreduce.am.command-opts
> yarn.app.mapreduce.am.env
> ACL related properties should go under OOZIE-2897:
> mapreduce.job.acl-view-job
> mapreduce.job.acl-modfy-job
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
Failed: OOZIE-3082 PreCommit Build #96
Jira: https://issues.apache.org/jira/browse/OOZIE-3082 Build: https://builds.apache.org/job/PreCommit-OOZIE-Build/96/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated 1.59 MB...] WARNING: the current HEAD has 77 Javadoc warning(s) +1 COMPILE +1 HEAD compiles +1 patch compiles +1 the patch does not seem to introduce new javac warnings -1 There are [1] new bugs found below threshold in total that must be fixed. +1 There are no new bugs found in [examples]. +1 There are no new bugs found in [core]. +1 There are no new bugs found in [sharelib/distcp]. +1 There are no new bugs found in [sharelib/hive]. +1 There are no new bugs found in [sharelib/pig]. +1 There are no new bugs found in [sharelib/spark]. +1 There are no new bugs found in [sharelib/hive2]. +1 There are no new bugs found in [sharelib/hcatalog]. +1 There are no new bugs found in [sharelib/sqoop]. -1 There are [1] new bugs found below threshold in [sharelib/oozie] that must be fixed. You can find the FindBugs diff here (look for the red and orange ones): sharelib/oozie/findbugs-new.html The most important FindBugs errors are: At ActionStats.java:[line 37]: Unwritten public or protected field: org.apache.oozie.action.hadoop.ActionStats.currentActionType +1 There are no new bugs found in [sharelib/streaming]. +1 There are no new bugs found in [tools]. +1 There are no new bugs found in [docs]. +1 There are no new bugs found in [server]. +1 There are no new bugs found in [client]. +1 BACKWARDS_COMPATIBILITY +1 the patch does not change any JPA Entity/Colum/Basic/Lob/Transient annotations +1 the patch does not modify JPA files +1 TESTS Tests run: 2039 Tests rerun: 57 Tests failed at first run: org.apache.oozie.command.bundle.TestBundleSubmitXCommand,org.apache.oozie.action.hadoop.TestJavaActionExecutor, +1 DISTRO +1 distro tarball builds with the patch -1 Overall result, please check the reported -1(s) There is at least one warning, please check The full output of the test-patch run is available at https://builds.apache.org/job/PreCommit-OOZIE-Build/96/ % Total% Received % Xferd Average Speed TimeTime Time Current Dload Upload Total SpentLeft Speed 0 00 00 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 00 00 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 00 00 0 0 0 --:--:-- --:--:-- --:--:-- 0100 3706k 100 3706k0 0 3658k 0 0:00:01 0:00:01 --:--:-- 7625k Adding comment to JIRA Comment added. test-patch exit code: 1 Build step 'Execute shell' marked build as failure [description-setter] Description set: OOZIE-3082 Archiving artifacts [Fast Archiver] Compressed 1.80 MB of artifacts by 62.7% relative to #95 Recording test results Email was triggered for: Failure - Any Sending email for trigger: Failure - Any ### ## FAILED TESTS (if any) ## All tests passed
[jira] [Commented] (OOZIE-3082) Move org.apache.oozie.action.hadoop.ActionStats to Oozie ShareLib
[
https://issues.apache.org/jira/browse/OOZIE-3082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16198877#comment-16198877
]
Hadoop QA commented on OOZIE-3082:
--
Testing JIRA OOZIE-3082
Cleaning local git workspace
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:red}-1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than
132
.{color:red}-1{color} the patch does not add/modify any testcase
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
warnings
.{color:red}WARNING{color}: the current HEAD has 77 Javadoc warning(s)
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac
warnings
{color:red}-1{color} There are [1] new bugs found below threshold in total that
must be fixed.
. {color:green}+1{color} There are no new bugs found in [examples].
. {color:green}+1{color} There are no new bugs found in [core].
. {color:green}+1{color} There are no new bugs found in [sharelib/distcp].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive].
. {color:green}+1{color} There are no new bugs found in [sharelib/pig].
. {color:green}+1{color} There are no new bugs found in [sharelib/spark].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive2].
. {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
. {color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
. {color:red}-1{color} There are [1] new bugs found below threshold in
[sharelib/oozie] that must be fixed.
. You can find the FindBugs diff here (look for the red and orange ones):
sharelib/oozie/findbugs-new.html
. The most important FindBugs errors are:
. At ActionStats.java:[line 37]: Unwritten public or protected field:
org.apache.oozie.action.hadoop.ActionStats.currentActionType
. {color:green}+1{color} There are no new bugs found in [sharelib/streaming].
. {color:green}+1{color} There are no new bugs found in [tools].
. {color:green}+1{color} There are no new bugs found in [docs].
. {color:green}+1{color} There are no new bugs found in [server].
. {color:green}+1{color} There are no new bugs found in [client].
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:green}+1 TESTS{color}
.Tests run: 2039
.Tests rerun: 57
.Tests failed at first run:
org.apache.oozie.command.bundle.TestBundleSubmitXCommand,org.apache.oozie.action.hadoop.TestJavaActionExecutor,
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch
{color:red}*-1 Overall result, please check the reported -1(s)*{color}
{color:red}. There is at least one warning, please check{color}
The full output of the test-patch run is available at
. https://builds.apache.org/job/PreCommit-OOZIE-Build/96/
> Move org.apache.oozie.action.hadoop.ActionStats to Oozie ShareLib
> -
>
> Key: OOZIE-3082
> URL: https://issues.apache.org/jira/browse/OOZIE-3082
> Project: Oozie
> Issue Type: Bug
>Affects Versions: 5.0.0
>Reporter: Attila Sasvari
>Assignee: Attila Sasvari
> Attachments: OOZIE-3082-001.patch
>
>
> OOZIE-2852 moved ActionStats to oozie-core. Because of this Pig jobs are
> failing with {{java.lang.NoClassDefFoundError:
> org/apache/oozie/action/hadoop/ActionStats}}
> Reproduction:
> - I executed the pig wf example on a cluster with {{bin/oozie job -oozie
> http://localhost:11000/oozie -config examples/apps/pig/job.properties -run
> -DnameNode=hdfs://localhost:9000 -DjobTracker=localhost:8032}}. RM web UI
> show the OoziLauncher failed:
> {code}
> State:FINISHED
> FinalStatus: FAILED
> Started: 10-Oct-2017 13:54:33
> Elapsed: 14sec
> Tracking URL: History
> Diagnostics: org/apache/oozie/action/hadoop/ActionStats
> {code}
> - Looking at the logs it turns out ActionStats is not contained anymore in
> the Oozie sharelib:
> {code}
> java.lang.NoClassDefFoundError: org/apache/oozie/action/hadoop/ActionStats
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:
[jira] [Updated] (OOZIE-3081) testSubmitWithLauncherQueue fails when FairScheduler is used
[
https://issues.apache.org/jira/browse/OOZIE-3081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Sasvari updated OOZIE-3081:
--
Attachment: OOZIE-3081-002.patch
[~pbacsko] thanks for the review, attached patch that addresses your comments.
> testSubmitWithLauncherQueue fails when FairScheduler is used
>
>
> Key: OOZIE-3081
> URL: https://issues.apache.org/jira/browse/OOZIE-3081
> Project: Oozie
> Issue Type: Bug
> Components: tests
>Affects Versions: 5.0.0
>Reporter: Attila Sasvari
>Assignee: Attila Sasvari
>Priority: Trivial
> Attachments: OOZIE-3081-001.patch, OOZIE-3081-002.patch
>
>
> Running TestJavaActionExecutor fails with fair scheduler
> {code}
> Failed tests:
> testSubmitWithLauncherQueue(org.apache.oozie.action.hadoop.TestJavaActionExecutor):
>
> expected:<[]test> but was:<[root.]test>
> {code}
> Reason: FairScheduler adds a "root." prefix to the queue name in
> [org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.QueueManager.ensureRootPrefix()|https://github.com/apache/hadoop/blob/99292adcefdc6b8f280b8e100605fb39f755c38a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/QueueManager.java#L165]
> A potential way to fix the test case:
> - we can add a new function
> {code}
> private boolean isFairSchedulerUsed(Configuration conf) {
> return
> conf.get("yarn.resourcemanager.scheduler.class").contains(FairScheduler.class.getName());
> }
> {code}
> that can be called from {{testSubmitWithLauncherQueue()}}:
> {code}
> ...
> if (isFairSchedulerUsed(conf)) {
> assertEquals("root.test", queue);
> }
> else {
> assertEquals("test", queue);
> }
> ...
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (OOZIE-2852) Remove simple-json dependency from oozie sharelib
[
https://issues.apache.org/jira/browse/OOZIE-2852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16198696#comment-16198696
]
Attila Sasvari commented on OOZIE-2852:
---
This breaks the pig action. I have opened OOZIE-3082 to address it.
> Remove simple-json dependency from oozie sharelib
> -
>
> Key: OOZIE-2852
> URL: https://issues.apache.org/jira/browse/OOZIE-2852
> Project: Oozie
> Issue Type: Bug
>Affects Versions: oya
>Reporter: Peter Cseh
>Assignee: Artem Ervits
> Fix For: oya, 5.0.0
>
> Attachments: OOZIE-2852-2.patch
>
>
> {{MRStats}} is the only class that needs json-simple in the oozie sharelib.
> It's only used by SqoopAcitonExecutor and MRActionExecutor. It should be
> moved to oozie-core so the json-simple depencency can be removed from the
> oozie sharelib.
> This is affecting only the oya branch right now.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (OOZIE-3082) Move org.apache.oozie.action.hadoop.ActionStats to Oozie ShareLib
[
https://issues.apache.org/jira/browse/OOZIE-3082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16198695#comment-16198695
]
Peter Cseh commented on OOZIE-3082:
---
Ohh, I haven't tested with Pig action when we moved this out from the
oozie-sharelib.
Fortunately, the dependency json-simple can still be avoided in the
oozie-sharelib.
+1 (pending jenkins)
> Move org.apache.oozie.action.hadoop.ActionStats to Oozie ShareLib
> -
>
> Key: OOZIE-3082
> URL: https://issues.apache.org/jira/browse/OOZIE-3082
> Project: Oozie
> Issue Type: Bug
>Affects Versions: 5.0.0
>Reporter: Attila Sasvari
>Assignee: Attila Sasvari
> Attachments: OOZIE-3082-001.patch
>
>
> OOZIE-2852 moved ActionStats to oozie-core. Because of this Pig jobs are
> failing with {{java.lang.NoClassDefFoundError:
> org/apache/oozie/action/hadoop/ActionStats}}
> Reproduction:
> - I executed the pig wf example on a cluster with {{bin/oozie job -oozie
> http://localhost:11000/oozie -config examples/apps/pig/job.properties -run
> -DnameNode=hdfs://localhost:9000 -DjobTracker=localhost:8032}}. RM web UI
> show the OoziLauncher failed:
> {code}
> State:FINISHED
> FinalStatus: FAILED
> Started: 10-Oct-2017 13:54:33
> Elapsed: 14sec
> Tracking URL: History
> Diagnostics: org/apache/oozie/action/hadoop/ActionStats
> {code}
> - Looking at the logs it turns out ActionStats is not contained anymore in
> the Oozie sharelib:
> {code}
> java.lang.NoClassDefFoundError: org/apache/oozie/action/hadoop/ActionStats
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:348)
> at
> org.apache.hadoop.conf.Configuration.getClassByNameOrNull(Configuration.java:2013)
> at
> org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1978)
> at
> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2072)
> at
> org.apache.oozie.action.hadoop.LauncherAM.runActionMain(LauncherAM.java:404)
> at
> org.apache.oozie.action.hadoop.LauncherAM.access$300(LauncherAM.java:56)
> at
> org.apache.oozie.action.hadoop.LauncherAM$2.run(LauncherAM.java:223)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
> at org.apache.oozie.action.hadoop.LauncherAM.run(LauncherAM.java:217)
> at
> org.apache.oozie.action.hadoop.LauncherAM$1.run(LauncherAM.java:153)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
> at org.apache.oozie.action.hadoop.LauncherAM.main(LauncherAM.java:140)
> Caused by: java.lang.ClassNotFoundException:
> org.apache.oozie.action.hadoop.ActionStats
> at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> ... 17 more
> {code}
> We should move ActionStats to sharelib (that is oozie-core's compile
> dependency).
> ( It might also make sense to create a docker based job to run the example
> workflows regularly to protect against this kind of failures in the future.
> Docker image could contain a simple, 1-node pseudo hadoop cluster and Oozie. )
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (OOZIE-2004) Improve Oozie version info output
[
https://issues.apache.org/jira/browse/OOZIE-2004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16198692#comment-16198692
]
Peter Cseh commented on OOZIE-2004:
---
Yes, that would be correct [~dbist13]. I've opened up OOZIE-3083 to track this.
> Improve Oozie version info output
> -
>
> Key: OOZIE-2004
> URL: https://issues.apache.org/jira/browse/OOZIE-2004
> Project: Oozie
> Issue Type: Improvement
> Components: build
>Affects Versions: 4.0.1
>Reporter: Chunjun Xiao
>Assignee: Artem Ervits
>Priority: Minor
> Fix For: 5.0.0
>
> Attachments: HBASE-2004-2.patch, OOZIE-2004-0.patch,
> OOZIE-2004-1.patch, OOZIE-2004-2.patch
>
>
> Current oozie version info output is quite simple:
> {code}
> [root@svm5 ~] # oozie version
> Oozie client build version: 4.0.1
> [root@svm5 ~] #
> {code}
> Can we improve it to follow the convention of Hadoop/HBase, as:
> {code}
> [root@svm5 ~] # oozie version
> Oozie 4.0.1
> Source code repository: ssh://git@XXX/oozie.git -r
> 3171cb656a4e4c1b2c1e9c6a47abed7df5bbcc8d
> Compiled by root on Sun Jul 6 04:38:08 PDT 2014
> From source with checksum XXX
> [root@svm5 ~] #
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (OOZIE-3083) Make improved version Info backward compatible
Peter Cseh created OOZIE-3083:
-
Summary: Make improved version Info backward compatible
Key: OOZIE-3083
URL: https://issues.apache.org/jira/browse/OOZIE-3083
Project: Oozie
Issue Type: Bug
Reporter: Peter Cseh
OOZIE-2004 changed the format of the version info output considerably.
We should change it back to a more compatible format.
It could be something like
{code}
{"buildVersion":"5.0.0-SNAPSHOT", "buildInfo":"{build.time=${build.time},
vc.url=${vc.url}, build.user=asasvari, vc.revision=${vc.revision}"}
{code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (OOZIE-3082) Move org.apache.oozie.action.hadoop.ActionStats to Oozie ShareLib
[
https://issues.apache.org/jira/browse/OOZIE-3082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Sasvari updated OOZIE-3082:
--
Attachment: OOZIE-3082-001.patch
> Move org.apache.oozie.action.hadoop.ActionStats to Oozie ShareLib
> -
>
> Key: OOZIE-3082
> URL: https://issues.apache.org/jira/browse/OOZIE-3082
> Project: Oozie
> Issue Type: Bug
>Affects Versions: 5.0.0
>Reporter: Attila Sasvari
>Assignee: Attila Sasvari
> Attachments: OOZIE-3082-001.patch
>
>
> OOZIE-2852 moved ActionStats to oozie-core. Because of this Pig jobs are
> failing with {{java.lang.NoClassDefFoundError:
> org/apache/oozie/action/hadoop/ActionStats}}
> Reproduction:
> - I executed the pig wf example on a cluster with {{bin/oozie job -oozie
> http://localhost:11000/oozie -config examples/apps/pig/job.properties -run
> -DnameNode=hdfs://localhost:9000 -DjobTracker=localhost:8032}}. RM web UI
> show the OoziLauncher failed:
> {code}
> State:FINISHED
> FinalStatus: FAILED
> Started: 10-Oct-2017 13:54:33
> Elapsed: 14sec
> Tracking URL: History
> Diagnostics: org/apache/oozie/action/hadoop/ActionStats
> {code}
> - Looking at the logs it turns out ActionStats is not contained anymore in
> the Oozie sharelib:
> {code}
> java.lang.NoClassDefFoundError: org/apache/oozie/action/hadoop/ActionStats
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:348)
> at
> org.apache.hadoop.conf.Configuration.getClassByNameOrNull(Configuration.java:2013)
> at
> org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1978)
> at
> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2072)
> at
> org.apache.oozie.action.hadoop.LauncherAM.runActionMain(LauncherAM.java:404)
> at
> org.apache.oozie.action.hadoop.LauncherAM.access$300(LauncherAM.java:56)
> at
> org.apache.oozie.action.hadoop.LauncherAM$2.run(LauncherAM.java:223)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
> at org.apache.oozie.action.hadoop.LauncherAM.run(LauncherAM.java:217)
> at
> org.apache.oozie.action.hadoop.LauncherAM$1.run(LauncherAM.java:153)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
> at org.apache.oozie.action.hadoop.LauncherAM.main(LauncherAM.java:140)
> Caused by: java.lang.ClassNotFoundException:
> org.apache.oozie.action.hadoop.ActionStats
> at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> ... 17 more
> {code}
> We should move ActionStats to sharelib (that is oozie-core's compile
> dependency).
> ( It might also make sense to create a docker based job to run the example
> workflows regularly to protect against this kind of failures in the future.
> Docker image could contain a simple, 1-node pseudo hadoop cluster and Oozie. )
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (OOZIE-3082) Move org.apache.oozie.action.hadoop.ActionStats to Oozie ShareLib
[
https://issues.apache.org/jira/browse/OOZIE-3082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Sasvari updated OOZIE-3082:
--
Attachment: (was: OOZIE-3082-001.patch)
> Move org.apache.oozie.action.hadoop.ActionStats to Oozie ShareLib
> -
>
> Key: OOZIE-3082
> URL: https://issues.apache.org/jira/browse/OOZIE-3082
> Project: Oozie
> Issue Type: Bug
>Affects Versions: 5.0.0
>Reporter: Attila Sasvari
>Assignee: Attila Sasvari
>
> OOZIE-2852 moved ActionStats to oozie-core. Because of this Pig jobs are
> failing with {{java.lang.NoClassDefFoundError:
> org/apache/oozie/action/hadoop/ActionStats}}
> Reproduction:
> - I executed the pig wf example on a cluster with {{bin/oozie job -oozie
> http://localhost:11000/oozie -config examples/apps/pig/job.properties -run
> -DnameNode=hdfs://localhost:9000 -DjobTracker=localhost:8032}}. RM web UI
> show the OoziLauncher failed:
> {code}
> State:FINISHED
> FinalStatus: FAILED
> Started: 10-Oct-2017 13:54:33
> Elapsed: 14sec
> Tracking URL: History
> Diagnostics: org/apache/oozie/action/hadoop/ActionStats
> {code}
> - Looking at the logs it turns out ActionStats is not contained anymore in
> the Oozie sharelib:
> {code}
> java.lang.NoClassDefFoundError: org/apache/oozie/action/hadoop/ActionStats
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:348)
> at
> org.apache.hadoop.conf.Configuration.getClassByNameOrNull(Configuration.java:2013)
> at
> org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1978)
> at
> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2072)
> at
> org.apache.oozie.action.hadoop.LauncherAM.runActionMain(LauncherAM.java:404)
> at
> org.apache.oozie.action.hadoop.LauncherAM.access$300(LauncherAM.java:56)
> at
> org.apache.oozie.action.hadoop.LauncherAM$2.run(LauncherAM.java:223)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
> at org.apache.oozie.action.hadoop.LauncherAM.run(LauncherAM.java:217)
> at
> org.apache.oozie.action.hadoop.LauncherAM$1.run(LauncherAM.java:153)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
> at org.apache.oozie.action.hadoop.LauncherAM.main(LauncherAM.java:140)
> Caused by: java.lang.ClassNotFoundException:
> org.apache.oozie.action.hadoop.ActionStats
> at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> ... 17 more
> {code}
> We should move ActionStats to sharelib (that is oozie-core's compile
> dependency).
> ( It might also make sense to create a docker based job to run the example
> workflows regularly to protect against this kind of failures in the future.
> Docker image could contain a simple, 1-node pseudo hadoop cluster and Oozie. )
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (OOZIE-3082) Move org.apache.oozie.action.hadoop.ActionStats to Oozie ShareLib
[
https://issues.apache.org/jira/browse/OOZIE-3082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Sasvari updated OOZIE-3082:
--
Attachment: OOZIE-3082-001.patch
> Move org.apache.oozie.action.hadoop.ActionStats to Oozie ShareLib
> -
>
> Key: OOZIE-3082
> URL: https://issues.apache.org/jira/browse/OOZIE-3082
> Project: Oozie
> Issue Type: Bug
>Affects Versions: 5.0.0
>Reporter: Attila Sasvari
>
> OOZIE-2852 moved ActionStats to oozie-core. Because of this Pig jobs are
> failing with {{java.lang.NoClassDefFoundError:
> org/apache/oozie/action/hadoop/ActionStats}}
> Reproduction:
> - I executed the pig wf example on a cluster with {{bin/oozie job -oozie
> http://localhost:11000/oozie -config examples/apps/pig/job.properties -run
> -DnameNode=hdfs://localhost:9000 -DjobTracker=localhost:8032}}. RM web UI
> show the OoziLauncher failed:
> {code}
> State:FINISHED
> FinalStatus: FAILED
> Started: 10-Oct-2017 13:54:33
> Elapsed: 14sec
> Tracking URL: History
> Diagnostics: org/apache/oozie/action/hadoop/ActionStats
> {code}
> - Looking at the logs it turns out ActionStats is not contained anymore in
> the Oozie sharelib:
> {code}
> java.lang.NoClassDefFoundError: org/apache/oozie/action/hadoop/ActionStats
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:348)
> at
> org.apache.hadoop.conf.Configuration.getClassByNameOrNull(Configuration.java:2013)
> at
> org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1978)
> at
> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2072)
> at
> org.apache.oozie.action.hadoop.LauncherAM.runActionMain(LauncherAM.java:404)
> at
> org.apache.oozie.action.hadoop.LauncherAM.access$300(LauncherAM.java:56)
> at
> org.apache.oozie.action.hadoop.LauncherAM$2.run(LauncherAM.java:223)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
> at org.apache.oozie.action.hadoop.LauncherAM.run(LauncherAM.java:217)
> at
> org.apache.oozie.action.hadoop.LauncherAM$1.run(LauncherAM.java:153)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
> at org.apache.oozie.action.hadoop.LauncherAM.main(LauncherAM.java:140)
> Caused by: java.lang.ClassNotFoundException:
> org.apache.oozie.action.hadoop.ActionStats
> at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> ... 17 more
> {code}
> We should move ActionStats to sharelib (that is oozie-core's compile
> dependency).
> ( It might also make sense to create a docker based job to run the example
> workflows regularly to protect against this kind of failures in the future.
> Docker image could contain a simple, 1-node pseudo hadoop cluster and Oozie. )
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Assigned] (OOZIE-3082) Move org.apache.oozie.action.hadoop.ActionStats to Oozie ShareLib
[
https://issues.apache.org/jira/browse/OOZIE-3082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Sasvari reassigned OOZIE-3082:
-
Assignee: Attila Sasvari
> Move org.apache.oozie.action.hadoop.ActionStats to Oozie ShareLib
> -
>
> Key: OOZIE-3082
> URL: https://issues.apache.org/jira/browse/OOZIE-3082
> Project: Oozie
> Issue Type: Bug
>Affects Versions: 5.0.0
>Reporter: Attila Sasvari
>Assignee: Attila Sasvari
>
> OOZIE-2852 moved ActionStats to oozie-core. Because of this Pig jobs are
> failing with {{java.lang.NoClassDefFoundError:
> org/apache/oozie/action/hadoop/ActionStats}}
> Reproduction:
> - I executed the pig wf example on a cluster with {{bin/oozie job -oozie
> http://localhost:11000/oozie -config examples/apps/pig/job.properties -run
> -DnameNode=hdfs://localhost:9000 -DjobTracker=localhost:8032}}. RM web UI
> show the OoziLauncher failed:
> {code}
> State:FINISHED
> FinalStatus: FAILED
> Started: 10-Oct-2017 13:54:33
> Elapsed: 14sec
> Tracking URL: History
> Diagnostics: org/apache/oozie/action/hadoop/ActionStats
> {code}
> - Looking at the logs it turns out ActionStats is not contained anymore in
> the Oozie sharelib:
> {code}
> java.lang.NoClassDefFoundError: org/apache/oozie/action/hadoop/ActionStats
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:348)
> at
> org.apache.hadoop.conf.Configuration.getClassByNameOrNull(Configuration.java:2013)
> at
> org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1978)
> at
> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2072)
> at
> org.apache.oozie.action.hadoop.LauncherAM.runActionMain(LauncherAM.java:404)
> at
> org.apache.oozie.action.hadoop.LauncherAM.access$300(LauncherAM.java:56)
> at
> org.apache.oozie.action.hadoop.LauncherAM$2.run(LauncherAM.java:223)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
> at org.apache.oozie.action.hadoop.LauncherAM.run(LauncherAM.java:217)
> at
> org.apache.oozie.action.hadoop.LauncherAM$1.run(LauncherAM.java:153)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
> at org.apache.oozie.action.hadoop.LauncherAM.main(LauncherAM.java:140)
> Caused by: java.lang.ClassNotFoundException:
> org.apache.oozie.action.hadoop.ActionStats
> at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> ... 17 more
> {code}
> We should move ActionStats to sharelib (that is oozie-core's compile
> dependency).
> ( It might also make sense to create a docker based job to run the example
> workflows regularly to protect against this kind of failures in the future.
> Docker image could contain a simple, 1-node pseudo hadoop cluster and Oozie. )
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (OOZIE-3082) Move org.apache.oozie.action.hadoop.ActionStats to Oozie ShareLib
Attila Sasvari created OOZIE-3082:
-
Summary: Move org.apache.oozie.action.hadoop.ActionStats to Oozie
ShareLib
Key: OOZIE-3082
URL: https://issues.apache.org/jira/browse/OOZIE-3082
Project: Oozie
Issue Type: Bug
Affects Versions: 5.0.0
Reporter: Attila Sasvari
OOZIE-2852 moved ActionStats to oozie-core. Because of this Pig jobs are
failing with {{java.lang.NoClassDefFoundError:
org/apache/oozie/action/hadoop/ActionStats}}
Reproduction:
- I executed the pig wf example on a cluster with {{bin/oozie job -oozie
http://localhost:11000/oozie -config examples/apps/pig/job.properties -run
-DnameNode=hdfs://localhost:9000 -DjobTracker=localhost:8032}}. RM web UI show
the OoziLauncher failed:
{code}
State: FINISHED
FinalStatus:FAILED
Started:10-Oct-2017 13:54:33
Elapsed:14sec
Tracking URL: History
Diagnostics:org/apache/oozie/action/hadoop/ActionStats
{code}
- Looking at the logs it turns out ActionStats is not contained anymore in the
Oozie sharelib:
{code}
java.lang.NoClassDefFoundError: org/apache/oozie/action/hadoop/ActionStats
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:348)
at
org.apache.hadoop.conf.Configuration.getClassByNameOrNull(Configuration.java:2013)
at
org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1978)
at
org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2072)
at
org.apache.oozie.action.hadoop.LauncherAM.runActionMain(LauncherAM.java:404)
at
org.apache.oozie.action.hadoop.LauncherAM.access$300(LauncherAM.java:56)
at org.apache.oozie.action.hadoop.LauncherAM$2.run(LauncherAM.java:223)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.oozie.action.hadoop.LauncherAM.run(LauncherAM.java:217)
at org.apache.oozie.action.hadoop.LauncherAM$1.run(LauncherAM.java:153)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.oozie.action.hadoop.LauncherAM.main(LauncherAM.java:140)
Caused by: java.lang.ClassNotFoundException:
org.apache.oozie.action.hadoop.ActionStats
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 17 more
{code}
We should move ActionStats to sharelib (that is oozie-core's compile
dependency).
( It might also make sense to create a docker based job to run the example
workflows regularly to protect against this kind of failures in the future.
Docker image could contain a simple, 1-node pseudo hadoop cluster and Oozie. )
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (OOZIE-3081) testSubmitWithLauncherQueue fails when FairScheduler is used
[
https://issues.apache.org/jira/browse/OOZIE-3081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16198532#comment-16198532
]
Peter Bacsko commented on OOZIE-3081:
-
Looks good. Two things:
1. Weird formatting:
{noformat}
+else {
+assertEquals("test", queue);
+}}
{noformat}
2. If it's possible to import, you can use the constant
{{org.apache.hadoop.yarn.conf.YarnConfiguration.RM_SCHEDULER}} in order to
avoid the property being hard-coded.
> testSubmitWithLauncherQueue fails when FairScheduler is used
>
>
> Key: OOZIE-3081
> URL: https://issues.apache.org/jira/browse/OOZIE-3081
> Project: Oozie
> Issue Type: Bug
> Components: tests
>Affects Versions: 5.0.0
>Reporter: Attila Sasvari
>Assignee: Attila Sasvari
>Priority: Trivial
> Attachments: OOZIE-3081-001.patch
>
>
> Running TestJavaActionExecutor fails with fair scheduler
> {code}
> Failed tests:
> testSubmitWithLauncherQueue(org.apache.oozie.action.hadoop.TestJavaActionExecutor):
>
> expected:<[]test> but was:<[root.]test>
> {code}
> Reason: FairScheduler adds a "root." prefix to the queue name in
> [org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.QueueManager.ensureRootPrefix()|https://github.com/apache/hadoop/blob/99292adcefdc6b8f280b8e100605fb39f755c38a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/QueueManager.java#L165]
> A potential way to fix the test case:
> - we can add a new function
> {code}
> private boolean isFairSchedulerUsed(Configuration conf) {
> return
> conf.get("yarn.resourcemanager.scheduler.class").contains(FairScheduler.class.getName());
> }
> {code}
> that can be called from {{testSubmitWithLauncherQueue()}}:
> {code}
> ...
> if (isFairSchedulerUsed(conf)) {
> assertEquals("root.test", queue);
> }
> else {
> assertEquals("test", queue);
> }
> ...
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
Success: OOZIE-3081 PreCommit Build #95
Jira: https://issues.apache.org/jira/browse/OOZIE-3081 Build: https://builds.apache.org/job/PreCommit-OOZIE-Build/95/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated 1.59 MB...] +1 RAT +1 the patch does not seem to introduce new RAT warnings +1 JAVADOC +1 the patch does not seem to introduce new Javadoc warnings WARNING: the current HEAD has 77 Javadoc warning(s) +1 COMPILE +1 HEAD compiles +1 patch compiles +1 the patch does not seem to introduce new javac warnings +1 There are no new bugs found in total. +1 There are no new bugs found in [docs]. +1 There are no new bugs found in [examples]. +1 There are no new bugs found in [tools]. +1 There are no new bugs found in [core]. +1 There are no new bugs found in [server]. +1 There are no new bugs found in [client]. +1 There are no new bugs found in [sharelib/distcp]. +1 There are no new bugs found in [sharelib/oozie]. +1 There are no new bugs found in [sharelib/hcatalog]. +1 There are no new bugs found in [sharelib/streaming]. +1 There are no new bugs found in [sharelib/hive]. +1 There are no new bugs found in [sharelib/sqoop]. +1 There are no new bugs found in [sharelib/spark]. +1 There are no new bugs found in [sharelib/hive2]. +1 There are no new bugs found in [sharelib/pig]. +1 BACKWARDS_COMPATIBILITY +1 the patch does not change any JPA Entity/Colum/Basic/Lob/Transient annotations +1 the patch does not modify JPA files +1 TESTS Tests run: 2039 Tests rerun: 72 Tests failed at first run: org.apache.oozie.command.coord.TestCoordActionInputCheckXCommand,org.apache.oozie.action.hadoop.TestJavaActionExecutor, +1 DISTRO +1 distro tarball builds with the patch +1 Overall result, good!, no -1s There is at least one warning, please check The full output of the test-patch run is available at https://builds.apache.org/job/PreCommit-OOZIE-Build/95/ % Total% Received % Xferd Average Speed TimeTime Time Current Dload Upload Total SpentLeft Speed 0 00 00 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 00 00 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 00 00 0 0 0 --:--:-- --:--:-- --:--:-- 0100 3706k 100 3706k0 0 4708k 0 --:--:-- --:--:-- --:--:-- 11.1M Adding comment to JIRA Comment added. test-patch exit code: 0 [description-setter] Description set: OOZIE-3081 Archiving artifacts [Fast Archiver] Compressed 1.79 MB of artifacts by 64.4% relative to #93 Recording test results Email was triggered for: Success Sending email for trigger: Success ### ## FAILED TESTS (if any) ## All tests passed
[jira] [Commented] (OOZIE-3081) testSubmitWithLauncherQueue fails when FairScheduler is used
[
https://issues.apache.org/jira/browse/OOZIE-3081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16198522#comment-16198522
]
Hadoop QA commented on OOZIE-3081:
--
Testing JIRA OOZIE-3081
Cleaning local git workspace
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:green}+1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than
132
.{color:green}+1{color} the patch does adds/modifies 1 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
warnings
.{color:red}WARNING{color}: the current HEAD has 77 Javadoc warning(s)
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac
warnings
{color:green}+1{color} There are no new bugs found in total.
. {color:green}+1{color} There are no new bugs found in [docs].
. {color:green}+1{color} There are no new bugs found in [examples].
. {color:green}+1{color} There are no new bugs found in [tools].
. {color:green}+1{color} There are no new bugs found in [core].
. {color:green}+1{color} There are no new bugs found in [server].
. {color:green}+1{color} There are no new bugs found in [client].
. {color:green}+1{color} There are no new bugs found in [sharelib/distcp].
. {color:green}+1{color} There are no new bugs found in [sharelib/oozie].
. {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
. {color:green}+1{color} There are no new bugs found in [sharelib/streaming].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive].
. {color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
. {color:green}+1{color} There are no new bugs found in [sharelib/spark].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive2].
. {color:green}+1{color} There are no new bugs found in [sharelib/pig].
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:green}+1 TESTS{color}
.Tests run: 2039
.Tests rerun: 72
.Tests failed at first run:
org.apache.oozie.command.coord.TestCoordActionInputCheckXCommand,org.apache.oozie.action.hadoop.TestJavaActionExecutor,
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch
{color:green}*+1 Overall result, good!, no -1s*{color}
{color:red}. There is at least one warning, please check{color}
The full output of the test-patch run is available at
. https://builds.apache.org/job/PreCommit-OOZIE-Build/95/
> testSubmitWithLauncherQueue fails when FairScheduler is used
>
>
> Key: OOZIE-3081
> URL: https://issues.apache.org/jira/browse/OOZIE-3081
> Project: Oozie
> Issue Type: Bug
> Components: tests
>Affects Versions: 5.0.0
>Reporter: Attila Sasvari
>Assignee: Attila Sasvari
>Priority: Trivial
> Attachments: OOZIE-3081-001.patch
>
>
> Running TestJavaActionExecutor fails with fair scheduler
> {code}
> Failed tests:
> testSubmitWithLauncherQueue(org.apache.oozie.action.hadoop.TestJavaActionExecutor):
>
> expected:<[]test> but was:<[root.]test>
> {code}
> Reason: FairScheduler adds a "root." prefix to the queue name in
> [org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.QueueManager.ensureRootPrefix()|https://github.com/apache/hadoop/blob/99292adcefdc6b8f280b8e100605fb39f755c38a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/QueueManager.java#L165]
> A potential way to fix the test case:
> - we can add a new function
> {code}
> private boolean isFairSchedulerUsed(Configuration conf) {
> return
> conf.get("yarn.resourcemanager.scheduler.class").contains(FairScheduler.class.getName());
> }
> {code}
> that can be called from {{testSubmitWithLauncherQueue()}}:
> {code}
> ...
> if (isFairSchedulerUsed(conf)) {
> assertEquals("root.test", queue);
> }
> else {
> assertEquals("test", queue);
> }
> ...
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (OOZIE-3081) testSubmitWithLauncherQueue fails when FairScheduler is used
[
https://issues.apache.org/jira/browse/OOZIE-3081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Sasvari updated OOZIE-3081:
--
Priority: Trivial (was: Major)
> testSubmitWithLauncherQueue fails when FairScheduler is used
>
>
> Key: OOZIE-3081
> URL: https://issues.apache.org/jira/browse/OOZIE-3081
> Project: Oozie
> Issue Type: Bug
> Components: tests
>Affects Versions: 5.0.0
>Reporter: Attila Sasvari
>Assignee: Attila Sasvari
>Priority: Trivial
>
> Running TestJavaActionExecutor fails with fair scheduler
> {code}
> Failed tests:
> testSubmitWithLauncherQueue(org.apache.oozie.action.hadoop.TestJavaActionExecutor):
>
> expected:<[]test> but was:<[root.]test>
> {code}
> Reason: FairScheduler adds a "root." prefix to the queue name in
> [org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.QueueManager.ensureRootPrefix()|https://github.com/apache/hadoop/blob/99292adcefdc6b8f280b8e100605fb39f755c38a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/QueueManager.java#L165]
> A potential way to fix the test case:
> - we can add a new function
> {code}
> private boolean isFairSchedulerUsed(Configuration conf) {
> return
> conf.get("yarn.resourcemanager.scheduler.class").contains(FairScheduler.class.getName());
> }
> {code}
> that can be called from {{testSubmitWithLauncherQueue()}}:
> {code}
> ...
> if (isFairSchedulerUsed(conf)) {
> assertEquals("root.test", queue);
> }
> else {
> assertEquals("test", queue);
> }
> ...
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Assigned] (OOZIE-3081) testSubmitWithLauncherQueue fails when FairScheduler is used
[
https://issues.apache.org/jira/browse/OOZIE-3081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Sasvari reassigned OOZIE-3081:
-
Assignee: Attila Sasvari
> testSubmitWithLauncherQueue fails when FairScheduler is used
>
>
> Key: OOZIE-3081
> URL: https://issues.apache.org/jira/browse/OOZIE-3081
> Project: Oozie
> Issue Type: Bug
> Components: tests
>Affects Versions: 5.0.0
>Reporter: Attila Sasvari
>Assignee: Attila Sasvari
>
> Running TestJavaActionExecutor fails with fair scheduler
> {code}
> Failed tests:
> testSubmitWithLauncherQueue(org.apache.oozie.action.hadoop.TestJavaActionExecutor):
>
> expected:<[]test> but was:<[root.]test>
> {code}
> Reason: FairScheduler adds a "root." prefix to the queue name in
> [org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.QueueManager.ensureRootPrefix()|https://github.com/apache/hadoop/blob/99292adcefdc6b8f280b8e100605fb39f755c38a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/QueueManager.java#L165]
> A potential way to fix the test case:
> - we can add a new function
> {code}
> private boolean isFairSchedulerUsed(Configuration conf) {
> return
> conf.get("yarn.resourcemanager.scheduler.class").contains(FairScheduler.class.getName());
> }
> {code}
> that can be called from {{testSubmitWithLauncherQueue()}}:
> {code}
> ...
> if (isFairSchedulerUsed(conf)) {
> assertEquals("root.test", queue);
> }
> else {
> assertEquals("test", queue);
> }
> ...
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (OOZIE-3081) testSubmitWithLauncherQueue fails when FairScheduler is used
[
https://issues.apache.org/jira/browse/OOZIE-3081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Sasvari updated OOZIE-3081:
--
Attachment: OOZIE-3081-001.patch
> testSubmitWithLauncherQueue fails when FairScheduler is used
>
>
> Key: OOZIE-3081
> URL: https://issues.apache.org/jira/browse/OOZIE-3081
> Project: Oozie
> Issue Type: Bug
> Components: tests
>Affects Versions: 5.0.0
>Reporter: Attila Sasvari
>Assignee: Attila Sasvari
>Priority: Trivial
> Attachments: OOZIE-3081-001.patch
>
>
> Running TestJavaActionExecutor fails with fair scheduler
> {code}
> Failed tests:
> testSubmitWithLauncherQueue(org.apache.oozie.action.hadoop.TestJavaActionExecutor):
>
> expected:<[]test> but was:<[root.]test>
> {code}
> Reason: FairScheduler adds a "root." prefix to the queue name in
> [org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.QueueManager.ensureRootPrefix()|https://github.com/apache/hadoop/blob/99292adcefdc6b8f280b8e100605fb39f755c38a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/QueueManager.java#L165]
> A potential way to fix the test case:
> - we can add a new function
> {code}
> private boolean isFairSchedulerUsed(Configuration conf) {
> return
> conf.get("yarn.resourcemanager.scheduler.class").contains(FairScheduler.class.getName());
> }
> {code}
> that can be called from {{testSubmitWithLauncherQueue()}}:
> {code}
> ...
> if (isFairSchedulerUsed(conf)) {
> assertEquals("root.test", queue);
> }
> else {
> assertEquals("test", queue);
> }
> ...
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
