date:20171207

Re: Review Request 64408: OOZIE-2869 amend Ability to add/remove MapReduce jars from action classpath

2017-12-07 Thread Peter Bacsko via Review Board


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64408/#review193130
---




core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java
Line 1564 (original), 1568 (patched)


Is it possible to test this somehow? I know it might not be easy because 
it's a private method... But still, if there's a way to do it, I'd appreciate 
it.


- Peter Bacsko


On dec. 7, 2017, 10 de, András Piros wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64408/
> ---
> 
> (Updated dec. 7, 2017, 10 de)
> 
> 
> Review request for oozie and Peter Cseh.
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> OOZIE-2869 amend Ability to add/remove MapReduce jars from action classpath
> 
> On real clusters, `launcherConf` might not be populated. We fall back to 
> `ConfigurationService.get().getConf()` in such cases.
> Also, by default Hive2 and Java actions should also load MapReduce sharelib 
> files.
> 
> 
> Diffs
> -
> 
>   core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 
> cab0d8b314cee3d42ee312aeeeccaf10529908d4 
>   core/src/main/java/org/apache/oozie/service/ConfigurationService.java 
> 4707252bef3a232a6e815360a18fe440e16ebdcc 
>   core/src/main/resources/oozie-default.xml 
> 7ec64bc9955b5cd5ce52e25785096fff0c7d3cca 
> 
> 
> Diff: https://reviews.apache.org/r/64408/diff/1/
> 
> 
> Testing
> ---
> 
> All relevant action types on a real cluster.
> 
> 
> Thanks,
> 
> András Piros
> 
>

[jira] [Commented] (OOZIE-3141) Expose external child job IDs to EL functions: wf:externalChildIDs(String actionName)

2017-12-07 Thread H Lu (JIRA)


[ 
https://issues.apache.org/jira/browse/OOZIE-3141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16282462#comment-16282462
 ] 

H Lu commented on OOZIE-3141:
-

[~mayank_bansal] Can you help review? Thanks.

> Expose external child job IDs to EL functions: wf:externalChildIDs(String 
> actionName)
> -
>
> Key: OOZIE-3141
> URL: https://issues.apache.org/jira/browse/OOZIE-3141
> Project: Oozie
>  Issue Type: Improvement
>  Components: core
>Affects Versions: 5.0.0b1
>Reporter: H Lu
>Priority: Minor
> Attachments: OOZIE-3141.0001.patch
>
>
> Expose external child job IDs to EL functions: wf:externalChildIDs(String 
> actionName).
> So we can use this EL function wf:externalChildIDs(String actionName) to get 
> the child jobs and call yarn rest api to get the job information such 
> diagnostics and embed the info in the email notifications. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (OOZIE-3141) Expose external child job IDs to EL functions: wf:externalChildIDs(String actionName)

2017-12-07 Thread H Lu (JIRA)


 [ 
https://issues.apache.org/jira/browse/OOZIE-3141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

H Lu updated OOZIE-3141:

Attachment: OOZIE-3141.0001.patch

> Expose external child job IDs to EL functions: wf:externalChildIDs(String 
> actionName)
> -
>
> Key: OOZIE-3141
> URL: https://issues.apache.org/jira/browse/OOZIE-3141
> Project: Oozie
>  Issue Type: Improvement
>  Components: core
>Affects Versions: 5.0.0b1
>Reporter: H Lu
>Priority: Minor
> Attachments: OOZIE-3141.0001.patch
>
>
> Expose external child job IDs to EL functions: wf:externalChildIDs(String 
> actionName).
> So we can use this EL function wf:externalChildIDs(String actionName) to get 
> the child jobs and call yarn rest api to get the job information such 
> diagnostics and embed the info in the email notifications. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (OOZIE-3141) Expose external child job IDs to EL functions: wf:externalChildIDs(String actionName)

2017-12-07 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/OOZIE-3141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16282426#comment-16282426
 ] 

ASF GitHub Bot commented on OOZIE-3141:
---

GitHub user pkuwm opened a pull request:

https://github.com/apache/oozie/pull/33

[OOZIE-3141] Expose external child job IDs to EL functions: wf:extern…

…alChildIDs(String actionName).

https://issues.apache.org/jira/browse/OOZIE-3141
Expose external child job IDs to EL functions: wf:externalChildIDs(String 
actionName).
So we can use this EL function wf:externalChildIDs(String actionName) to 
get the child jobs and call yarn rest api to get the job information such 
diagnostics and embed the info in the email notifications.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/pkuwm/oozie OOZIE-3141

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/oozie/pull/33.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #33


commit a570c6caace0b750224d12171eaa070b23a0cfcf
Author: pkuwm 
Date:   2017-12-07T19:52:52Z

[OOZIE-3141] Expose external child job IDs to EL functions: 
wf:externalChildIDs(String actionName).




> Expose external child job IDs to EL functions: wf:externalChildIDs(String 
> actionName)
> -
>
> Key: OOZIE-3141
> URL: https://issues.apache.org/jira/browse/OOZIE-3141
> Project: Oozie
>  Issue Type: Improvement
>  Components: core
>Affects Versions: 5.0.0b1
>Reporter: H Lu
>Priority: Minor
>
> Expose external child job IDs to EL functions: wf:externalChildIDs(String 
> actionName).
> So we can use this EL function wf:externalChildIDs(String actionName) to get 
> the child jobs and call yarn rest api to get the job information such 
> diagnostics and embed the info in the email notifications. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] oozie pull request #33: [OOZIE-3141] Expose external child job IDs to EL fun...

2017-12-07 Thread pkuwm

GitHub user pkuwm opened a pull request:

https://github.com/apache/oozie/pull/33

[OOZIE-3141] Expose external child job IDs to EL functions: wf:externâ¦

â¦alChildIDs(String actionName).

https://issues.apache.org/jira/browse/OOZIE-3141
Expose external child job IDs to EL functions: wf:externalChildIDs(String 
actionName).
So we can use this EL function wf:externalChildIDs(String actionName) to 
get the child jobs and call yarn rest api to get the job information such 
diagnostics and embed the info in the email notifications.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/pkuwm/oozie OOZIE-3141

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/oozie/pull/33.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #33


commit a570c6caace0b750224d12171eaa070b23a0cfcf
Author: pkuwm 
Date:   2017-12-07T19:52:52Z

[OOZIE-3141] Expose external child job IDs to EL functions: 
wf:externalChildIDs(String actionName).




---

Success: OOZIE-3139 PreCommit Build #279

2017-12-07 Thread Apache Jenkins Server

Jira: https://issues.apache.org/jira/browse/OOZIE-3139
Build: https://builds.apache.org/job/PreCommit-OOZIE-Build/279/

###
## LAST 100 LINES OF THE CONSOLE 
###
[...truncated 1.59 MB...]
[DEBUG] There are no new bugs found in [sharelib/sqoop].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [sharelib/oozie].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [sharelib/pig].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [sharelib/hive].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [sharelib/spark].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [client].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [examples].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [docs].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [server].
[INFO] There are no new bugs found totally].
[TRACE] FindBugs diffs checked and reports created
[TRACE] Summary file size is 2115 bytes
[TRACE] Full summary file size is 1248 bytes
[TRACE] File 
[/home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/tmp/FINDBUGS_DIFF/diff/findbugs-diff-0.1.0-all.jar]
 removed
[TRACE] File 
[/home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/tmp/FINDBUGS_DIFF/diff/findbugs-diff-0.1.0-all.jar.md5sum]
 removed
  Running test-patch task BACKWARDS_COMPATIBILITY
  Running test-patch task TESTS
  Running test-patch task DISTRO


Testing JIRA OOZIE-3139

Cleaning local git workspace



+1 PATCH_APPLIES
+1 CLEAN
+1 RAW_PATCH_ANALYSIS
+1 the patch does not introduce any @author tags
+1 the patch does not introduce any tabs
+1 the patch does not introduce any trailing spaces
+1 the patch does not introduce any line longer than 132
+1 the patch does adds/modifies 3 testcase(s)
+1 RAT
+1 the patch does not seem to introduce new RAT warnings
+1 JAVADOC
+1 the patch does not seem to introduce new Javadoc warnings
+1 COMPILE
+1 HEAD compiles
+1 patch compiles
+1 the patch does not seem to introduce new javac warnings
+1 There are no new bugs found in total.
 +1 There are no new bugs found in [core].
 +1 There are no new bugs found in [tools].
 +1 There are no new bugs found in [sharelib/hive2].
 +1 There are no new bugs found in [sharelib/distcp].
 +1 There are no new bugs found in [sharelib/hcatalog].
 +1 There are no new bugs found in [sharelib/streaming].
 +1 There are no new bugs found in [sharelib/sqoop].
 +1 There are no new bugs found in [sharelib/oozie].
 +1 There are no new bugs found in [sharelib/pig].
 +1 There are no new bugs found in [sharelib/hive].
 +1 There are no new bugs found in [sharelib/spark].
 +1 There are no new bugs found in [client].
 +1 There are no new bugs found in [examples].
 +1 There are no new bugs found in [docs].
 +1 There are no new bugs found in [server].
+1 BACKWARDS_COMPATIBILITY
+1 the patch does not change any JPA Entity/Colum/Basic/Lob/Transient 
annotations
+1 the patch does not modify JPA files
+1 TESTS
Tests run: 2077
Tests rerun: 57
Tests failed at first run: 
org.apache.oozie.action.hadoop.TestJavaActionExecutor,org.apache.oozie.command.coord.TestCoordActionsKillXCommand,
+1 DISTRO
+1 distro tarball builds with the patch 


+1 Overall result, good!, no -1s


The full output of the test-patch run is available at

 https://builds.apache.org/job/PreCommit-OOZIE-Build/279/

  % Total% Received % Xferd  Average Speed   TimeTime Time  Current
 Dload  Upload   Total   SpentLeft  Speed
  0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 0  
0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 0
  0 3706k0  86880 0  10688  0  0:05:55 --:--:--  0:05:55 
10688100 3706k  100 3706k0 0  3589k  0  0:00:01  0:00:01 --:--:-- 
16.4M
Adding comment to JIRA
Comment added.

test-patch exit code: 0

[description-setter] Description set: OOZIE-3139
Archiving artifacts
[Fast Archiver] Compressed 1.71 MB of artifacts by 36.5% relative to #268
Recording test results
Email was triggered for: Success
Sending email for trigger: Success



###
## FAILED TESTS (if any)

Re: Review Request 64408: OOZIE-2869 amend Ability to add/remove MapReduce jars from action classpath

2017-12-07 Thread Peter Bacsko via Review Board


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64408/#review193132
---




core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java
Line 1559 (original), 1559 (patched)


This statement is no longer valid: "Subclasses should override this method 
if necessary"


- Peter Bacsko


On dec. 7, 2017, 10 de, András Piros wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64408/
> ---
> 
> (Updated dec. 7, 2017, 10 de)
> 
> 
> Review request for oozie and Peter Cseh.
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> OOZIE-2869 amend Ability to add/remove MapReduce jars from action classpath
> 
> On real clusters, `launcherConf` might not be populated. We fall back to 
> `ConfigurationService.get().getConf()` in such cases.
> Also, by default Hive2 and Java actions should also load MapReduce sharelib 
> files.
> 
> 
> Diffs
> -
> 
>   core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 
> cab0d8b314cee3d42ee312aeeeccaf10529908d4 
>   core/src/main/java/org/apache/oozie/service/ConfigurationService.java 
> 4707252bef3a232a6e815360a18fe440e16ebdcc 
>   core/src/main/resources/oozie-default.xml 
> 7ec64bc9955b5cd5ce52e25785096fff0c7d3cca 
> 
> 
> Diff: https://reviews.apache.org/r/64408/diff/1/
> 
> 
> Testing
> ---
> 
> All relevant action types on a real cluster.
> 
> 
> Thanks,
> 
> András Piros
> 
>

[jira] [Commented] (OOZIE-2869) Ability to add/remove MapReduce jars from action classpath.

2017-12-07 Thread Andras Piros (JIRA)


[ 
https://issues.apache.org/jira/browse/OOZIE-2869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16282119#comment-16282119
 ] 

Andras Piros commented on OOZIE-2869:
-

Thanks [~gezapeti] for the review! Committed amendment patch to master.

> Ability to add/remove MapReduce jars from action classpath.
> ---
>
> Key: OOZIE-2869
> URL: https://issues.apache.org/jira/browse/OOZIE-2869
> Project: Oozie
>  Issue Type: Sub-task
>  Components: action
>Affects Versions: 5.0.0
>Reporter: Andras Piros
>Assignee: Andras Piros
> Fix For: 5.0.0b1
>
> Attachments: OOZIE-2869.001.patch, OOZIE-2869.amend.001.patch
>
>
> Since Oozie On YARN doesn't fire a map only job to fire all the other 
> applications (hence the transition from {{LauncherMapper}} to 
> {{LauncherAM}}), no need to bundle the MapReduce JARs within Oozie's default 
> sharelib.
> Nevertheless, it's a common, but discouraged use case that MapReduce tasks 
> are fired using Java Action. So we need some default {{mapreduce}} sharelib 
> that can be used by customers after OOZIE-2601 will have been implemented:
> # by default, create a sharelib named {{mapreduce}} which will contain the MR 
> jars using local paths as those jars are on every machine
> # users can add mapreduce jars to any action using 
> {{oozie.action.sharelib.for.ACTION}} just like now. They could add MR jars 
> into Java, Spark or Shell actions easily



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (OOZIE-2869) Ability to add/remove MapReduce jars from action classpath.

2017-12-07 Thread Peter Cseh (JIRA)


[ 
https://issues.apache.org/jira/browse/OOZIE-2869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16282040#comment-16282040
 ] 

Peter Cseh commented on OOZIE-2869:
---

Good catch! +1 for the amend-patch

> Ability to add/remove MapReduce jars from action classpath.
> ---
>
> Key: OOZIE-2869
> URL: https://issues.apache.org/jira/browse/OOZIE-2869
> Project: Oozie
>  Issue Type: Sub-task
>  Components: action
>Affects Versions: 5.0.0
>Reporter: Andras Piros
>Assignee: Andras Piros
> Fix For: 5.0.0b1
>
> Attachments: OOZIE-2869.001.patch, OOZIE-2869.amend.001.patch
>
>
> Since Oozie On YARN doesn't fire a map only job to fire all the other 
> applications (hence the transition from {{LauncherMapper}} to 
> {{LauncherAM}}), no need to bundle the MapReduce JARs within Oozie's default 
> sharelib.
> Nevertheless, it's a common, but discouraged use case that MapReduce tasks 
> are fired using Java Action. So we need some default {{mapreduce}} sharelib 
> that can be used by customers after OOZIE-2601 will have been implemented:
> # by default, create a sharelib named {{mapreduce}} which will contain the MR 
> jars using local paths as those jars are on every machine
> # users can add mapreduce jars to any action using 
> {{oozie.action.sharelib.for.ACTION}} just like now. They could add MR jars 
> into Java, Spark or Shell actions easily



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (OOZIE-2869) Ability to add/remove MapReduce jars from action classpath.

2017-12-07 Thread Andras Piros (JIRA)


[ 
https://issues.apache.org/jira/browse/OOZIE-2869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16281936#comment-16281936
 ] 

Andras Piros commented on OOZIE-2869:
-

[~gezapeti] [~pbacsko] can you please review over RB the amendment patch? 
Thanks!

> Ability to add/remove MapReduce jars from action classpath.
> ---
>
> Key: OOZIE-2869
> URL: https://issues.apache.org/jira/browse/OOZIE-2869
> Project: Oozie
>  Issue Type: Sub-task
>  Components: action
>Affects Versions: 5.0.0
>Reporter: Andras Piros
>Assignee: Andras Piros
> Fix For: 5.0.0b1
>
> Attachments: OOZIE-2869.001.patch, OOZIE-2869.amend.001.patch
>
>
> Since Oozie On YARN doesn't fire a map only job to fire all the other 
> applications (hence the transition from {{LauncherMapper}} to 
> {{LauncherAM}}), no need to bundle the MapReduce JARs within Oozie's default 
> sharelib.
> Nevertheless, it's a common, but discouraged use case that MapReduce tasks 
> are fired using Java Action. So we need some default {{mapreduce}} sharelib 
> that can be used by customers after OOZIE-2601 will have been implemented:
> # by default, create a sharelib named {{mapreduce}} which will contain the MR 
> jars using local paths as those jars are on every machine
> # users can add mapreduce jars to any action using 
> {{oozie.action.sharelib.for.ACTION}} just like now. They could add MR jars 
> into Java, Spark or Shell actions easily



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (OOZIE-2869) Ability to add/remove MapReduce jars from action classpath.

2017-12-07 Thread Hadoop QA (JIRA)


[ 
https://issues.apache.org/jira/browse/OOZIE-2869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16281856#comment-16281856
 ] 

Hadoop QA commented on OOZIE-2869:
--

Testing JIRA OOZIE-2869

Cleaning local git workspace



{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:red}-1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than 
132
.{color:red}-1{color} the patch does not add/modify any testcase
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT 
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc 
warnings
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac 
warnings
{color:green}+1{color} There are no new bugs found in total.
. {color:green}+1{color} There are no new bugs found in [core].
. {color:green}+1{color} There are no new bugs found in [tools].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive2].
. {color:green}+1{color} There are no new bugs found in [sharelib/distcp].
. {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
. {color:green}+1{color} There are no new bugs found in [sharelib/streaming].
. {color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
. {color:green}+1{color} There are no new bugs found in [sharelib/oozie].
. {color:green}+1{color} There are no new bugs found in [sharelib/pig].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive].
. {color:green}+1{color} There are no new bugs found in [sharelib/spark].
. {color:green}+1{color} There are no new bugs found in [client].
. {color:green}+1{color} There are no new bugs found in [examples].
. {color:green}+1{color} There are no new bugs found in [docs].
. {color:green}+1{color} There are no new bugs found in [server].
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA 
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:green}+1 TESTS{color}
.Tests run: 2075
.Tests rerun: 106
.Tests failed at first run: 
org.apache.oozie.action.hadoop.TestJavaActionExecutor,org.apache.oozie.service.TestPartitionDependencyManagerService,org.apache.oozie.service.TestJMSAccessorService,org.apache.oozie.service.TestStatusTransitService,org.apache.oozie.command.coord.TestCoordActionsKillXCommand,
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch 


{color:red}*-1 Overall result, please check the reported -1(s)*{color}


The full output of the test-patch run is available at

. https://builds.apache.org/job/PreCommit-OOZIE-Build/278/

> Ability to add/remove MapReduce jars from action classpath.
> ---
>
> Key: OOZIE-2869
> URL: https://issues.apache.org/jira/browse/OOZIE-2869
> Project: Oozie
>  Issue Type: Sub-task
>  Components: action
>Affects Versions: 5.0.0
>Reporter: Andras Piros
>Assignee: Andras Piros
> Fix For: 5.0.0b1
>
> Attachments: OOZIE-2869.001.patch, OOZIE-2869.amend.001.patch
>
>
> Since Oozie On YARN doesn't fire a map only job to fire all the other 
> applications (hence the transition from {{LauncherMapper}} to 
> {{LauncherAM}}), no need to bundle the MapReduce JARs within Oozie's default 
> sharelib.
> Nevertheless, it's a common, but discouraged use case that MapReduce tasks 
> are fired using Java Action. So we need some default {{mapreduce}} sharelib 
> that can be used by customers after OOZIE-2601 will have been implemented:
> # by default, create a sharelib named {{mapreduce}} which will contain the MR 
> jars using local paths as those jars are on every machine
> # users can add mapreduce jars to any action using 
> {{oozie.action.sharelib.for.ACTION}} just like now. They could add MR jars 
> into Java, Spark or Shell actions easily



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Failed: OOZIE-2869 PreCommit Build #278

2017-12-07 Thread Apache Jenkins Server

Jira: https://issues.apache.org/jira/browse/OOZIE-2869
Build: https://builds.apache.org/job/PreCommit-OOZIE-Build/278/

###
## LAST 100 LINES OF THE CONSOLE 
###
[...truncated 1.59 MB...]
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [sharelib/oozie].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [sharelib/pig].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [sharelib/hive].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [sharelib/spark].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [client].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [examples].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [docs].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [server].
[INFO] There are no new bugs found totally].
[TRACE] FindBugs diffs checked and reports created
[TRACE] Summary file size is 2111 bytes
[TRACE] Full summary file size is 1248 bytes
[TRACE] File 
[/home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/tmp/FINDBUGS_DIFF/diff/findbugs-diff-0.1.0-all.jar]
 removed
[TRACE] File 
[/home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/tmp/FINDBUGS_DIFF/diff/findbugs-diff-0.1.0-all.jar.md5sum]
 removed
  Running test-patch task BACKWARDS_COMPATIBILITY
  Running test-patch task TESTS
  Running test-patch task DISTRO


Testing JIRA OOZIE-2869

Cleaning local git workspace



+1 PATCH_APPLIES
+1 CLEAN
-1 RAW_PATCH_ANALYSIS
+1 the patch does not introduce any @author tags
+1 the patch does not introduce any tabs
+1 the patch does not introduce any trailing spaces
+1 the patch does not introduce any line longer than 132
-1 the patch does not add/modify any testcase
+1 RAT
+1 the patch does not seem to introduce new RAT warnings
+1 JAVADOC
+1 the patch does not seem to introduce new Javadoc warnings
+1 COMPILE
+1 HEAD compiles
+1 patch compiles
+1 the patch does not seem to introduce new javac warnings
+1 There are no new bugs found in total.
 +1 There are no new bugs found in [core].
 +1 There are no new bugs found in [tools].
 +1 There are no new bugs found in [sharelib/hive2].
 +1 There are no new bugs found in [sharelib/distcp].
 +1 There are no new bugs found in [sharelib/hcatalog].
 +1 There are no new bugs found in [sharelib/streaming].
 +1 There are no new bugs found in [sharelib/sqoop].
 +1 There are no new bugs found in [sharelib/oozie].
 +1 There are no new bugs found in [sharelib/pig].
 +1 There are no new bugs found in [sharelib/hive].
 +1 There are no new bugs found in [sharelib/spark].
 +1 There are no new bugs found in [client].
 +1 There are no new bugs found in [examples].
 +1 There are no new bugs found in [docs].
 +1 There are no new bugs found in [server].
+1 BACKWARDS_COMPATIBILITY
+1 the patch does not change any JPA Entity/Colum/Basic/Lob/Transient 
annotations
+1 the patch does not modify JPA files
+1 TESTS
Tests run: 2075
Tests rerun: 106
Tests failed at first run: 
org.apache.oozie.action.hadoop.TestJavaActionExecutor,org.apache.oozie.service.TestPartitionDependencyManagerService,org.apache.oozie.service.TestJMSAccessorService,org.apache.oozie.service.TestStatusTransitService,org.apache.oozie.command.coord.TestCoordActionsKillXCommand,
+1 DISTRO
+1 distro tarball builds with the patch 


-1 Overall result, please check the reported -1(s)


The full output of the test-patch run is available at

 https://builds.apache.org/job/PreCommit-OOZIE-Build/278/

  % Total% Received % Xferd  Average Speed   TimeTime Time  Current
 Dload  Upload   Total   SpentLeft  Speed
  0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 0  
0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 0  0 
00 00 0  0  0 --:--:-- --:--:-- --:--:-- 0
  1 3706k1 405440 0  39506  0  0:01:36  0:00:01  0:01:35 
39506100 3706k  100 3706k0 0  3098k  0  0:00:01  0:00:01 --:--:-- 
21.1M
Adding comment to JIRA
Comment added.

test-patch exit code: 1

Build step 'Execute shell' marked build as failure
[description-setter] Description set: OOZIE-2869
Archiving artifacts
[Fast Archiver] Compressed 1.71 MB of artifacts by 25.5%

[jira] [Comment Edited] (OOZIE-3139) Oozie validates workflow incorrectly

2017-12-07 Thread Peter Bacsko (JIRA)


[ 
https://issues.apache.org/jira/browse/OOZIE-3139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16280108#comment-16280108
 ] 

Peter Bacsko edited comment on OOZIE-3139 at 12/7/17 1:15 PM:
--

Sorry I my suggestion above was incorrect - you're not building the WF from 
objects, instead it's loaded from an XML, so it doesn't work.

However, I'd like to ask one final (minor) change in the two new tests:

{noformat}
LiteWorkflowApp app = 
parser.validateAndParse(IOUtils.getResourceAsReader(
   "wf-multiple-error-parent.xml", -1), new Configuration());
{noformat}

The {{app}} variable is unused, therefore it generates a warning. Just drop the 
variable declaration and ignore the return value of "parser".


was (Author: pbacsko):
Sorry I my suggestion above was incorrect - you're not building the WF from 
objects, instead it's loaded from an XML, so it doesn't work.

However, I'd like to ask one final (minor) change in the two new tests:

{noformat}
LiteWorkflowApp app = 
parser.validateAndParse(IOUtils.getResourceAsReader(
   "wf-multiple-error-parent.xml", -1), new Configuration());
{noformat]

The {{app}} variable is unused, therefore it generates a warning. Just drop the 
variable declaration and ignore the return value of "parser".

> Oozie validates workflow incorrectly
> 
>
> Key: OOZIE-3139
> URL: https://issues.apache.org/jira/browse/OOZIE-3139
> Project: Oozie
>  Issue Type: Bug
>  Components: core
>Affects Versions: 4.3.0
>Reporter: Denes Bodo
>Assignee: Denes Bodo
>Priority: Critical
> Attachments: OOZIE-3139.001.patch, OOZIE-3139.002.patch, 
> OOZIE-3139.003.patch, OOZIE-3139.004.patch, OOZIE-3139.005.patch, 
> exception.log, workflow.pdf
>
>
> We found that after OOZIE-1978 the attached workflow becomes invalid. Aa we 
> found out the following line brings the problem: 
> https://github.com/apache/oozie/commit/8e9b9042b3270dc5ff975c44a5c977fcc41250e4#diff-a38ae78886b7c3afb711d32cfb4dbbbdR267
>  . The parameter "okPath" changed from okPath to true.
> The following workflow is problematic:
> {noformat}
> 
> 
> 
> 
> 
> 
> 
> 
> ${jobTracker}
> ${nameNode}
> ${myscript}
> ${myscriptPath}
> 
> 
> 
> 
> 
> 
> 
> ${jobTracker}
> ${nameNode}
> ${myscript}
> ${myscriptPath}
> 
> 
> 
> 
> 
> 
> 
> 
> ${jobTracker}
> ${nameNode}
> ${myscript}
> ${myscriptPath}
> 
> 
> 
> 
> 
> 
> 
> ${jobTracker}
> ${nameNode}
> ${myscript}
> ${myscriptPath}
> 
> 
> 
> 
> 
> 
> 
> ${jobTracker}
> ${nameNode}
> ${myscript}
> ${myscriptPath}
> 
> 
> 
> 
>  
>
> 
> Shell action failed, error 
> message[${wf:errorMessage(wf:lastErrorNode())}]
> 
> 
> 
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (OOZIE-3139) Oozie validates workflow incorrectly

2017-12-07 Thread Denes Bodo (JIRA)


 [ 
https://issues.apache.org/jira/browse/OOZIE-3139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Denes Bodo updated OOZIE-3139:
--
Attachment: OOZIE-3139.005.patch

Add missing Apache license headers.

> Oozie validates workflow incorrectly
> 
>
> Key: OOZIE-3139
> URL: https://issues.apache.org/jira/browse/OOZIE-3139
> Project: Oozie
>  Issue Type: Bug
>  Components: core
>Affects Versions: 4.3.0
>Reporter: Denes Bodo
>Assignee: Denes Bodo
>Priority: Critical
> Attachments: OOZIE-3139.001.patch, OOZIE-3139.002.patch, 
> OOZIE-3139.003.patch, OOZIE-3139.004.patch, OOZIE-3139.005.patch, 
> exception.log, workflow.pdf
>
>
> We found that after OOZIE-1978 the attached workflow becomes invalid. Aa we 
> found out the following line brings the problem: 
> https://github.com/apache/oozie/commit/8e9b9042b3270dc5ff975c44a5c977fcc41250e4#diff-a38ae78886b7c3afb711d32cfb4dbbbdR267
>  . The parameter "okPath" changed from okPath to true.
> The following workflow is problematic:
> {noformat}
> 
> 
> 
> 
> 
> 
> 
> 
> ${jobTracker}
> ${nameNode}
> ${myscript}
> ${myscriptPath}
> 
> 
> 
> 
> 
> 
> 
> ${jobTracker}
> ${nameNode}
> ${myscript}
> ${myscriptPath}
> 
> 
> 
> 
> 
> 
> 
> 
> ${jobTracker}
> ${nameNode}
> ${myscript}
> ${myscriptPath}
> 
> 
> 
> 
> 
> 
> 
> ${jobTracker}
> ${nameNode}
> ${myscript}
> ${myscriptPath}
> 
> 
> 
> 
> 
> 
> 
> ${jobTracker}
> ${nameNode}
> ${myscript}
> ${myscriptPath}
> 
> 
> 
> 
>  
>
> 
> Shell action failed, error 
> message[${wf:errorMessage(wf:lastErrorNode())}]
> 
> 
> 
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (OOZIE-3139) Oozie validates workflow incorrectly

2017-12-07 Thread Hadoop QA (JIRA)


[ 
https://issues.apache.org/jira/browse/OOZIE-3139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16281763#comment-16281763
 ] 

Hadoop QA commented on OOZIE-3139:
--

Testing JIRA OOZIE-3139

Cleaning local git workspace



{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:green}+1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than 
132
.{color:green}+1{color} the patch does adds/modifies 3 testcase(s)
{color:red}-1 RAT{color}
.{color:red}-1{color} the patch seems to introduce 2 new RAT warning(s)
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc 
warnings
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac 
warnings
{color:green}+1{color} There are no new bugs found in total.
. {color:green}+1{color} There are no new bugs found in [docs].
. {color:green}+1{color} There are no new bugs found in [sharelib/distcp].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive].
. {color:green}+1{color} There are no new bugs found in [sharelib/spark].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive2].
. {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
. {color:green}+1{color} There are no new bugs found in [sharelib/streaming].
. {color:green}+1{color} There are no new bugs found in [sharelib/pig].
. {color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
. {color:green}+1{color} There are no new bugs found in [sharelib/oozie].
. {color:green}+1{color} There are no new bugs found in [examples].
. {color:green}+1{color} There are no new bugs found in [client].
. {color:green}+1{color} There are no new bugs found in [core].
. {color:green}+1{color} There are no new bugs found in [tools].
. {color:green}+1{color} There are no new bugs found in [server].
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA 
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:green}+1 TESTS{color}
.Tests run: 2077
.Tests rerun: 60
.Tests failed at first run: 
org.apache.oozie.action.hadoop.TestJavaActionExecutor,org.apache.oozie.service.TestCoordMaterializeTriggerService,
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch 


{color:red}*-1 Overall result, please check the reported -1(s)*{color}


The full output of the test-patch run is available at

. https://builds.apache.org/job/PreCommit-OOZIE-Build/277/

> Oozie validates workflow incorrectly
> 
>
> Key: OOZIE-3139
> URL: https://issues.apache.org/jira/browse/OOZIE-3139
> Project: Oozie
>  Issue Type: Bug
>  Components: core
>Affects Versions: 4.3.0
>Reporter: Denes Bodo
>Assignee: Denes Bodo
>Priority: Critical
> Attachments: OOZIE-3139.001.patch, OOZIE-3139.002.patch, 
> OOZIE-3139.003.patch, OOZIE-3139.004.patch, exception.log, workflow.pdf
>
>
> We found that after OOZIE-1978 the attached workflow becomes invalid. Aa we 
> found out the following line brings the problem: 
> https://github.com/apache/oozie/commit/8e9b9042b3270dc5ff975c44a5c977fcc41250e4#diff-a38ae78886b7c3afb711d32cfb4dbbbdR267
>  . The parameter "okPath" changed from okPath to true.
> The following workflow is problematic:
> {noformat}
> 
> 
> 
> 
> 
> 
> 
> 
> ${jobTracker}
> ${nameNode}
> ${myscript}
> ${myscriptPath}
> 
> 
> 
> 
> 
> 
> 
> ${jobTracker}
> ${nameNode}
> ${myscript}
> ${myscriptPath}
> 
> 
> 
> 
> 
> 
> 
> 
> ${jobTracker}
> ${nameNode}
> ${myscript}
> ${myscriptPath}
> 
> 
> 
> 
> 
> 
> 
> ${jobTracker}
> ${nameNode}
> ${myscript}
> ${myscriptPath}
> 
> 
> 
> 
> 
> 
> 
> ${jobTracker}
> ${nameNode}
> ${myscript}
> ${myscriptPath}
> 
> 
> 
> 
>

Failed: OOZIE-3139 PreCommit Build #277

2017-12-07 Thread Apache Jenkins Server

Jira: https://issues.apache.org/jira/browse/OOZIE-3139
Build: https://builds.apache.org/job/PreCommit-OOZIE-Build/277/

###
## LAST 100 LINES OF THE CONSOLE 
###
[...truncated 1.57 MB...]
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [sharelib/pig].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [sharelib/sqoop].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [sharelib/oozie].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [examples].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [client].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [core].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [tools].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [server].
[INFO] There are no new bugs found totally].
[TRACE] FindBugs diffs checked and reports created
[TRACE] Summary file size is 2107 bytes
[TRACE] Full summary file size is 1248 bytes
[TRACE] File 
[/home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/tmp/FINDBUGS_DIFF/diff/findbugs-diff-0.1.0-all.jar]
 removed
[TRACE] File 
[/home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/tmp/FINDBUGS_DIFF/diff/findbugs-diff-0.1.0-all.jar.md5sum]
 removed
  Running test-patch task BACKWARDS_COMPATIBILITY
  Running test-patch task TESTS
  Running test-patch task DISTRO


Testing JIRA OOZIE-3139

Cleaning local git workspace



+1 PATCH_APPLIES
+1 CLEAN
+1 RAW_PATCH_ANALYSIS
+1 the patch does not introduce any @author tags
+1 the patch does not introduce any tabs
+1 the patch does not introduce any trailing spaces
+1 the patch does not introduce any line longer than 132
+1 the patch does adds/modifies 3 testcase(s)
-1 RAT
-1 the patch seems to introduce 2 new RAT warning(s)
+1 JAVADOC
+1 the patch does not seem to introduce new Javadoc warnings
+1 COMPILE
+1 HEAD compiles
+1 patch compiles
+1 the patch does not seem to introduce new javac warnings
+1 There are no new bugs found in total.
 +1 There are no new bugs found in [docs].
 +1 There are no new bugs found in [sharelib/distcp].
 +1 There are no new bugs found in [sharelib/hive].
 +1 There are no new bugs found in [sharelib/spark].
 +1 There are no new bugs found in [sharelib/hive2].
 +1 There are no new bugs found in [sharelib/hcatalog].
 +1 There are no new bugs found in [sharelib/streaming].
 +1 There are no new bugs found in [sharelib/pig].
 +1 There are no new bugs found in [sharelib/sqoop].
 +1 There are no new bugs found in [sharelib/oozie].
 +1 There are no new bugs found in [examples].
 +1 There are no new bugs found in [client].
 +1 There are no new bugs found in [core].
 +1 There are no new bugs found in [tools].
 +1 There are no new bugs found in [server].
+1 BACKWARDS_COMPATIBILITY
+1 the patch does not change any JPA Entity/Colum/Basic/Lob/Transient 
annotations
+1 the patch does not modify JPA files
+1 TESTS
Tests run: 2077
Tests rerun: 60
Tests failed at first run: 
org.apache.oozie.action.hadoop.TestJavaActionExecutor,org.apache.oozie.service.TestCoordMaterializeTriggerService,
+1 DISTRO
+1 distro tarball builds with the patch 


-1 Overall result, please check the reported -1(s)


The full output of the test-patch run is available at

 https://builds.apache.org/job/PreCommit-OOZIE-Build/277/

  % Total% Received % Xferd  Average Speed   TimeTime Time  Current
 Dload  Upload   Total   SpentLeft  Speed
  0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 0  
0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 0  0 
00 00 0  0  0 --:--:-- --:--:-- --:--:-- 0
100 3706k  100 3706k0 0  3284k  0  0:00:01  0:00:01 --:--:-- 3284k
Adding comment to JIRA
Comment added.

test-patch exit code: 1

Build step 'Execute shell' marked build as failure
[description-setter] Description set: OOZIE-3139
Archiving artifacts
[Fast Archiver] Compressed 1.71 MB of artifacts by 36.4% relative to #268
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



###
## FAILED TESTS (if

[jira] [Updated] (OOZIE-2973) Make sure Oozie works with Hadoop 3

2017-12-07 Thread Andras Piros (JIRA)


 [ 
https://issues.apache.org/jira/browse/OOZIE-2973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andras Piros updated OOZIE-2973:

Affects Version/s: 4.3.0

> Make sure Oozie works with Hadoop 3 
> 
>
> Key: OOZIE-2973
> URL: https://issues.apache.org/jira/browse/OOZIE-2973
> Project: Oozie
>  Issue Type: Bug
>Affects Versions: 4.3.0
>Reporter: Attila Sasvari
>Assignee: Peter Cseh
>Priority: Blocker
> Fix For: 5.0.0
>
> Attachments: OOZIE-2973.001.patch, OOZIE-2973.002.patch
>
>
> This JIRA is to track that Oozie works with Hadoop 3. 
> - Build Oozie with latest hadoop 3 (e.g. assuming Apache Hadoop 3.0.0-alpha3 
> pass {{-D-Dhadoop.version=3.0.0-alpha3}} to maven) and execute tests
> - It would be nice to create an Apache jenkins job that runs Oozie tests with 
> hadoop 3 (also with other component) 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Review Request 64408: OOZIE-2869 amend Ability to add/remove MapReduce jars from action classpath

2017-12-07 Thread András Piros via Review Board


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64408/
---

Review request for oozie and Peter Cseh.


Repository: oozie-git


Description
---

OOZIE-2869 amend Ability to add/remove MapReduce jars from action classpath

On real clusters, `launcherConf` might not be populated. We fall back to 
`ConfigurationService.get().getConf()` in such cases.
Also, by default Hive2 and Java actions should also load MapReduce sharelib 
files.


Diffs
-

  core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 
cab0d8b314cee3d42ee312aeeeccaf10529908d4 
  core/src/main/java/org/apache/oozie/service/ConfigurationService.java 
4707252bef3a232a6e815360a18fe440e16ebdcc 
  core/src/main/resources/oozie-default.xml 
7ec64bc9955b5cd5ce52e25785096fff0c7d3cca 


Diff: https://reviews.apache.org/r/64408/diff/1/


Testing
---

All relevant action types on a real cluster.


Thanks,

András Piros

[jira] [Updated] (OOZIE-2869) Ability to add/remove MapReduce jars from action classpath.

2017-12-07 Thread Andras Piros (JIRA)


 [ 
https://issues.apache.org/jira/browse/OOZIE-2869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andras Piros updated OOZIE-2869:

Attachment: OOZIE-2869.amend.001.patch

> Ability to add/remove MapReduce jars from action classpath.
> ---
>
> Key: OOZIE-2869
> URL: https://issues.apache.org/jira/browse/OOZIE-2869
> Project: Oozie
>  Issue Type: Sub-task
>  Components: action
>Affects Versions: 5.0.0
>Reporter: Andras Piros
>Assignee: Andras Piros
> Fix For: 5.0.0b1
>
> Attachments: OOZIE-2869.001.patch, OOZIE-2869.amend.001.patch
>
>
> Since Oozie On YARN doesn't fire a map only job to fire all the other 
> applications (hence the transition from {{LauncherMapper}} to 
> {{LauncherAM}}), no need to bundle the MapReduce JARs within Oozie's default 
> sharelib.
> Nevertheless, it's a common, but discouraged use case that MapReduce tasks 
> are fired using Java Action. So we need some default {{mapreduce}} sharelib 
> that can be used by customers after OOZIE-2601 will have been implemented:
> # by default, create a sharelib named {{mapreduce}} which will contain the MR 
> jars using local paths as those jars are on every machine
> # users can add mapreduce jars to any action using 
> {{oozie.action.sharelib.for.ACTION}} just like now. They could add MR jars 
> into Java, Spark or Shell actions easily



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Reopened] (OOZIE-2869) Ability to add/remove MapReduce jars from action classpath.

2017-12-07 Thread Andras Piros (JIRA)


 [ 
https://issues.apache.org/jira/browse/OOZIE-2869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andras Piros reopened OOZIE-2869:
-
  Assignee: Andras Piros  (was: Peter Cseh)

On real clusters, {{launcherConf}} might not be populated. We fall back to 
{{ConfigurationService.get().getConf()}} in such cases.

Also, by default Hive2 and Java actions should also load MapReduce sharelib 
files.

> Ability to add/remove MapReduce jars from action classpath.
> ---
>
> Key: OOZIE-2869
> URL: https://issues.apache.org/jira/browse/OOZIE-2869
> Project: Oozie
>  Issue Type: Sub-task
>  Components: action
>Affects Versions: 5.0.0
>Reporter: Andras Piros
>Assignee: Andras Piros
> Fix For: 5.0.0b1
>
> Attachments: OOZIE-2869.001.patch
>
>
> Since Oozie On YARN doesn't fire a map only job to fire all the other 
> applications (hence the transition from {{LauncherMapper}} to 
> {{LauncherAM}}), no need to bundle the MapReduce JARs within Oozie's default 
> sharelib.
> Nevertheless, it's a common, but discouraged use case that MapReduce tasks 
> are fired using Java Action. So we need some default {{mapreduce}} sharelib 
> that can be used by customers after OOZIE-2601 will have been implemented:
> # by default, create a sharelib named {{mapreduce}} which will contain the MR 
> jars using local paths as those jars are on every machine
> # users can add mapreduce jars to any action using 
> {{oozie.action.sharelib.for.ACTION}} just like now. They could add MR jars 
> into Java, Spark or Shell actions easily



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: Review Request 64408: OOZIE-2869 amend Ability to add/remove MapReduce jars from action classpath

[jira] [Commented] (OOZIE-3141) Expose external child job IDs to EL functions: wf:externalChildIDs(String actionName)

[jira] [Updated] (OOZIE-3141) Expose external child job IDs to EL functions: wf:externalChildIDs(String actionName)

[jira] [Commented] (OOZIE-3141) Expose external child job IDs to EL functions: wf:externalChildIDs(String actionName)

[GitHub] oozie pull request #33: [OOZIE-3141] Expose external child job IDs to EL fun...

Success: OOZIE-3139 PreCommit Build #279

Re: Review Request 64408: OOZIE-2869 amend Ability to add/remove MapReduce jars from action classpath

[jira] [Commented] (OOZIE-2869) Ability to add/remove MapReduce jars from action classpath.

[jira] [Commented] (OOZIE-2869) Ability to add/remove MapReduce jars from action classpath.

[jira] [Commented] (OOZIE-2869) Ability to add/remove MapReduce jars from action classpath.

[jira] [Commented] (OOZIE-2869) Ability to add/remove MapReduce jars from action classpath.

Failed: OOZIE-2869 PreCommit Build #278

[jira] [Comment Edited] (OOZIE-3139) Oozie validates workflow incorrectly

[jira] [Updated] (OOZIE-3139) Oozie validates workflow incorrectly

[jira] [Commented] (OOZIE-3139) Oozie validates workflow incorrectly

Failed: OOZIE-3139 PreCommit Build #277

[jira] [Updated] (OOZIE-2973) Make sure Oozie works with Hadoop 3

Review Request 64408: OOZIE-2869 amend Ability to add/remove MapReduce jars from action classpath

[jira] [Updated] (OOZIE-2869) Ability to add/remove MapReduce jars from action classpath.

[jira] [Reopened] (OOZIE-2869) Ability to add/remove MapReduce jars from action classpath.

20 matches

Site Navigation

Mail list logo

Footer information