[jira] Subscription: Oozie Patch Available

2018-07-22 Thread jira 
Issue Subscription
Filter: Oozie Patch Available (95 issues)

Subscriber: ooziedaily

Key Summary
OOZIE-3304  Parsing sharelib timestamps is not threadsafe
https://issues.apache.org/jira/browse/OOZIE-3304
OOZIE-3301  Update NOTICE file
https://issues.apache.org/jira/browse/OOZIE-3301
OOZIE-3298  OYA: external ID is not filled properly and failing MR job is 
treated as SUCCEEDED
https://issues.apache.org/jira/browse/OOZIE-3298
OOZIE-3274  Remove slf4j
https://issues.apache.org/jira/browse/OOZIE-3274
OOZIE-3266  Coord action rerun support RERUN_SKIP_NODES option
https://issues.apache.org/jira/browse/OOZIE-3266
OOZIE-3265  properties RERUN_FAIL_NODES and RERUN_SKIP_NODES should be able to 
appear together
https://issues.apache.org/jira/browse/OOZIE-3265
OOZIE-3256  refactor OozieCLI class
https://issues.apache.org/jira/browse/OOZIE-3256
OOZIE-3249  [tools] Instrumentation log parser
https://issues.apache.org/jira/browse/OOZIE-3249
OOZIE-3218  Oozie Sqoop action with command splits the select clause into 
multiple parts due to delimiter being space
https://issues.apache.org/jira/browse/OOZIE-3218
OOZIE-3199  Let system property restriction configurable
https://issues.apache.org/jira/browse/OOZIE-3199
OOZIE-3196  Authorization: restrict world readability by user
https://issues.apache.org/jira/browse/OOZIE-3196
OOZIE-3194  Oozie should set proper permissions to sharelib after upload
https://issues.apache.org/jira/browse/OOZIE-3194
OOZIE-3193  Applications are not killed when submitted via subworkflow
https://issues.apache.org/jira/browse/OOZIE-3193
OOZIE-3186  Oozie is unable to use configuration linked using jceks://file/...
https://issues.apache.org/jira/browse/OOZIE-3186
OOZIE-3179  Adding a configurable config-default.xml location to a workflow
https://issues.apache.org/jira/browse/OOZIE-3179
OOZIE-3170  Oozie Diagnostic Bundle tool fails with NPE due to missing service 
class
https://issues.apache.org/jira/browse/OOZIE-3170
OOZIE-3160  PriorityDelayQueue put()/take() can cause significant CPU load due 
to busy waiting
https://issues.apache.org/jira/browse/OOZIE-3160
OOZIE-3156  Retry SSH action check when cannot connect to remote host
https://issues.apache.org/jira/browse/OOZIE-3156
OOZIE-3135  Configure log4j2 in SqoopMain
https://issues.apache.org/jira/browse/OOZIE-3135
OOZIE-3091  Oozie Sqoop Avro Import fails with "java.lang.NoClassDefFoundError: 
org/apache/avro/mapred/AvroWrapper"
https://issues.apache.org/jira/browse/OOZIE-3091
OOZIE-3071  Oozie 4.3 Spark sharelib ueses a different version of commons-lang3 
than Spark 2.2.0
https://issues.apache.org/jira/browse/OOZIE-3071
OOZIE-3063  Sanitizing variables that are part of openjpa.ConnectionProperties
https://issues.apache.org/jira/browse/OOZIE-3063
OOZIE-3062  Set HADOOP_CONF_DIR for spark action
https://issues.apache.org/jira/browse/OOZIE-3062
OOZIE-3061  Kill only those child jobs which are not already killed
https://issues.apache.org/jira/browse/OOZIE-3061
OOZIE-2952  Fix Findbugs warnings in oozie-sharelib-oozie
https://issues.apache.org/jira/browse/OOZIE-2952
OOZIE-2949  Escape quotes whitespaces in Sqoop  field
https://issues.apache.org/jira/browse/OOZIE-2949
OOZIE-2942  [examples] Fix Findbugs warnings
https://issues.apache.org/jira/browse/OOZIE-2942
OOZIE-2927  Append new line character for Hive2 query using query tag
https://issues.apache.org/jira/browse/OOZIE-2927
OOZIE-2877  Oozie Git Action
https://issues.apache.org/jira/browse/OOZIE-2877
OOZIE-2834  ParameterVerifier logging non-useful warning for workflow definition
https://issues.apache.org/jira/browse/OOZIE-2834
OOZIE-2833  when using uber mode the regex pattern used in the 
extractHeapSizeMB method does not allow heap sizes specified in bytes.
https://issues.apache.org/jira/browse/OOZIE-2833
OOZIE-2829  Improve sharelib upload to accept multiple source folders
https://issues.apache.org/jira/browse/OOZIE-2829
OOZIE-2812  SparkConfigurationService should support loading configurations 
from multiple Spark versions
https://issues.apache.org/jira/browse/OOZIE-2812
OOZIE-2795  Create lib directory or symlink for Oozie CLI during packaging
https://issues.apache.org/jira/browse/OOZIE-2795
OOZIE-2784  Include WEEK as a parameter in the Coordinator Expression Language 
Evaulator
https://issues.apache.org/jira/browse/OOZIE-2784
OOZIE-2779  Mask Hive2 action Beeline JDBC password
https://issues.apache.org/jira/browse/OOZIE-2779
OOZIE-2736  Reduce the number of threads during test execution
https://issues.apache.org/jira/browse/OOZIE-2736
OOZIE-2718  Im

Failed: OOZIE-3265 PreCommit Build #682

2018-07-22 Thread Apache Jenkins Server 
Jira: https://issues.apache.org/jira/browse/OOZIE-3265
Build: https://builds.apache.org/job/PreCommit-OOZIE-Build/682/

###
## LAST 100 LINES OF THE CONSOLE 
###
[...truncated 31.08 KB...]
at 
hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1741)
at 
hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357)
at hudson.remoting.Channel.call(Channel.java:955)
at 
org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.execute(RemoteGitImpl.java:146)
at sun.reflect.GeneratedMethodAccessor1143.invoke(Unknown 
Source)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.invoke(RemoteGitImpl.java:132)
at com.sun.proxy.$Proxy118.execute(Unknown Source)
at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:886)
at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1155)
at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1186)
at hudson.scm.SCM.checkout(SCM.java:504)
at 
hudson.model.AbstractProject.checkout(AbstractProject.java:1208)
at 
hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574)
at 
jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
at 
hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
at hudson.model.Run.execute(Run.java:1794)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at 
hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:429)
ERROR: Error fetching remote repo 'origin'
Retrying after 10 seconds
 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url 
 > https://git-wip-us.apache.org/repos/asf/oozie.git # timeout=10
Fetching upstream changes from https://git-wip-us.apache.org/repos/asf/oozie.git
 > git --version # timeout=10
 > git fetch --tags --progress 
 > https://git-wip-us.apache.org/repos/asf/oozie.git 
 > +refs/heads/*:refs/remotes/origin/*
ERROR: Error fetching remote repo 'origin'
hudson.plugins.git.GitException: Failed to fetch from 
https://git-wip-us.apache.org/repos/asf/oozie.git
at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:888)
at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1155)
at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1186)
at hudson.scm.SCM.checkout(SCM.java:504)
at hudson.model.AbstractProject.checkout(AbstractProject.java:1208)
at 
hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574)
at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
at 
hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
at hudson.model.Run.execute(Run.java:1794)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:429)
Caused by: hudson.plugins.git.GitException: Command "git fetch --tags 
--progress https://git-wip-us.apache.org/repos/asf/oozie.git 
+refs/heads/*:refs/remotes/origin/*" returned status code 128:
stdout: 
stderr: remote: Counting objects: 14689, done.
remote: Compressing objects:   0% (1/5983)   remote: Compressing 
objects:   1% (60/5983)   remote: Compressing objects:   2% (120/5983)  
 remote: Compressing objects:   3% (180/5983)   remote: 
Compressing objects:   4% (240/5983)   remote: Compressing objects:   
5% (300/5983)   remote: Compressing objects:   6% (359/5983)   
remote: Compressing objects:   7% (419/5983)   remote: Compressing 
objects:   8% (479/5983)   remote: Compressing objects:   9% (539/5983) 
  remote: Compressing objects:  10% (599/5983)   remote: 
Compressing objects:  11% (659/5983)   remote: Compressing objects:  
12% (718/5983)   remote: Compressing objects:  13% (778/5983)   
remote: Compressing objects:  14% (838/5983)   remote: Compressing 
objects:  15% (898/5983)   remote: Compressing objects:  16% (958/5983) 
  remote: Compressing objects:  17% (1018/5983)   remote: 
Compressing objects:  18% (1077/5983)   remote: Compressing objects:  
19% (1137/5983)   remote: Compressing objects:  20% (11

[jira] [Updated] (OOZIE-3265) properties RERUN_FAIL_NODES and RERUN_SKIP_NODES should be able to appear together

2018-07-22 Thread TIAN XING (JIRA) 


 [ 
https://issues.apache.org/jira/browse/OOZIE-3265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

TIAN XING updated OOZIE-3265:
-
Attachment: (was: OOZIE-3265-v3.patch)

> properties RERUN_FAIL_NODES and RERUN_SKIP_NODES should be able to appear 
> together
> --
>
> Key: OOZIE-3265
> URL: https://issues.apache.org/jira/browse/OOZIE-3265
> Project: Oozie
>  Issue Type: Task
>Affects Versions: 5.0.0
>Reporter: TIAN XING
>Assignee: TIAN XING
>Priority: Minor
> Attachments: OOZIE-3265-v1.patch, OOZIE-3265-v2.patch, rerun.patch
>
>
> Currently when you re-run a workflow with property "oozie.wf.rerun.failnodes" 
>  set to true,
> you can no longer re-run it again with "oozie.wf.rerun.skip.nodes" property 
> specified, even if you set "oozie.wf.rerun.failnodes" to false.
> This kind of limitation is not reasonable. There is only one case where 
> "oozie.wf.rerun.failnodes" is true and "oozie.wf.rerun.skip.nodes" is not 
> null or empty, that should be disallowed.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (OOZIE-3265) properties RERUN_FAIL_NODES and RERUN_SKIP_NODES should be able to appear together

2018-07-22 Thread TIAN XING (JIRA) 


 [ 
https://issues.apache.org/jira/browse/OOZIE-3265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

TIAN XING updated OOZIE-3265:
-
Attachment: (was: OOZIE-3265-v3.patch)

> properties RERUN_FAIL_NODES and RERUN_SKIP_NODES should be able to appear 
> together
> --
>
> Key: OOZIE-3265
> URL: https://issues.apache.org/jira/browse/OOZIE-3265
> Project: Oozie
>  Issue Type: Task
>Affects Versions: 5.0.0
>Reporter: TIAN XING
>Assignee: TIAN XING
>Priority: Minor
> Attachments: OOZIE-3265-v1.patch, OOZIE-3265-v2.patch, 
> OOZIE-3265-v3.patch, rerun.patch
>
>
> Currently when you re-run a workflow with property "oozie.wf.rerun.failnodes" 
>  set to true,
> you can no longer re-run it again with "oozie.wf.rerun.skip.nodes" property 
> specified, even if you set "oozie.wf.rerun.failnodes" to false.
> This kind of limitation is not reasonable. There is only one case where 
> "oozie.wf.rerun.failnodes" is true and "oozie.wf.rerun.skip.nodes" is not 
> null or empty, that should be disallowed.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (OOZIE-3265) properties RERUN_FAIL_NODES and RERUN_SKIP_NODES should be able to appear together

2018-07-22 Thread TIAN XING (JIRA) 


 [ 
https://issues.apache.org/jira/browse/OOZIE-3265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

TIAN XING updated OOZIE-3265:
-
Attachment: OOZIE-3265-v3.patch

> properties RERUN_FAIL_NODES and RERUN_SKIP_NODES should be able to appear 
> together
> --
>
> Key: OOZIE-3265
> URL: https://issues.apache.org/jira/browse/OOZIE-3265
> Project: Oozie
>  Issue Type: Task
>Affects Versions: 5.0.0
>Reporter: TIAN XING
>Assignee: TIAN XING
>Priority: Minor
> Attachments: OOZIE-3265-v1.patch, OOZIE-3265-v2.patch, 
> OOZIE-3265-v3.patch, rerun.patch
>
>
> Currently when you re-run a workflow with property "oozie.wf.rerun.failnodes" 
>  set to true,
> you can no longer re-run it again with "oozie.wf.rerun.skip.nodes" property 
> specified, even if you set "oozie.wf.rerun.failnodes" to false.
> This kind of limitation is not reasonable. There is only one case where 
> "oozie.wf.rerun.failnodes" is true and "oozie.wf.rerun.skip.nodes" is not 
> null or empty, that should be disallowed.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Failed: OOZIE-2829 PreCommit Build #681

2018-07-22 Thread Apache Jenkins Server 
Jira: https://issues.apache.org/jira/browse/OOZIE-2829
Build: https://builds.apache.org/job/PreCommit-OOZIE-Build/681/

###
## LAST 100 LINES OF THE CONSOLE 
###
[...truncated 31.08 KB...]
at 
hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1741)
at 
hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357)
at hudson.remoting.Channel.call(Channel.java:955)
at 
org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.execute(RemoteGitImpl.java:146)
at sun.reflect.GeneratedMethodAccessor1143.invoke(Unknown 
Source)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.invoke(RemoteGitImpl.java:132)
at com.sun.proxy.$Proxy118.execute(Unknown Source)
at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:886)
at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1155)
at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1186)
at hudson.scm.SCM.checkout(SCM.java:504)
at 
hudson.model.AbstractProject.checkout(AbstractProject.java:1208)
at 
hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574)
at 
jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
at 
hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
at hudson.model.Run.execute(Run.java:1794)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at 
hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:429)
ERROR: Error fetching remote repo 'origin'
Retrying after 10 seconds
 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url 
 > https://git-wip-us.apache.org/repos/asf/oozie.git # timeout=10
Fetching upstream changes from https://git-wip-us.apache.org/repos/asf/oozie.git
 > git --version # timeout=10
 > git fetch --tags --progress 
 > https://git-wip-us.apache.org/repos/asf/oozie.git 
 > +refs/heads/*:refs/remotes/origin/*
ERROR: Error fetching remote repo 'origin'
hudson.plugins.git.GitException: Failed to fetch from 
https://git-wip-us.apache.org/repos/asf/oozie.git
at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:888)
at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1155)
at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1186)
at hudson.scm.SCM.checkout(SCM.java:504)
at hudson.model.AbstractProject.checkout(AbstractProject.java:1208)
at 
hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574)
at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
at 
hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
at hudson.model.Run.execute(Run.java:1794)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:429)
Caused by: hudson.plugins.git.GitException: Command "git fetch --tags 
--progress https://git-wip-us.apache.org/repos/asf/oozie.git 
+refs/heads/*:refs/remotes/origin/*" returned status code 128:
stdout: 
stderr: remote: Counting objects: 14689, done.
remote: Compressing objects:   0% (1/5983)   remote: Compressing 
objects:   1% (60/5983)   remote: Compressing objects:   2% (120/5983)  
 remote: Compressing objects:   3% (180/5983)   remote: 
Compressing objects:   4% (240/5983)   remote: Compressing objects:   
5% (300/5983)   remote: Compressing objects:   6% (359/5983)   
remote: Compressing objects:   7% (419/5983)   remote: Compressing 
objects:   8% (479/5983)   remote: Compressing objects:   9% (539/5983) 
  remote: Compressing objects:  10% (599/5983)   remote: 
Compressing objects:  11% (659/5983)   remote: Compressing objects:  
12% (718/5983)   remote: Compressing objects:  13% (778/5983)   
remote: Compressing objects:  14% (838/5983)   remote: Compressing 
objects:  15% (898/5983)   remote: Compressing objects:  16% (958/5983) 
  remote: Compressing objects:  17% (1018/5983)   remote: 
Compressing objects:  18% (1077/5983)   remote: Compressing objects:  
19% (1137/5983)   remote: Compressing objects:  20% (11

Re: Review Request 67834: OOZIE-2829 Improve sharelib upload to accept multiple source folders

2018-07-22 Thread Kinga Marton via Review Board 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67834/
---

(Updated July 22, 2018, 4 p.m.)


Review request for oozie, András Piros, Peter Cseh, and Peter Bacsko.


Changes
---

test failure fix


Repository: oozie-git


Description
---

Right now sharelib can be created via sharelib create -fs FS_URI -locallib 
SHARED_LIBRARY where the SHARED_LIBRARY can be a tarbal or a folder.
It would be nice to have the possibility to define additional folders to be 
uploaded into the sharelib, so the users don't have to copy or link the files 
together on their machine.
The syntax could be something like -additional-lib 
sharelibName=/path/to/source/;/path/to/some/file,sharelibName2=/path/to/some/folder


Diffs (updated)
-

  docs/src/site/twiki/AG_Install.twiki 46363a3b3 
  tools/src/main/java/org/apache/oozie/tools/OozieSharelibCLI.java 75e932c02 
  
tools/src/test/java/org/apache/oozie/tools/IntegrationTestOozieSharelibCLI.java 
PRE-CREATION 
  tools/src/test/java/org/apache/oozie/tools/OozieSharelibFileOperations.java 
d344300ed 
  tools/src/test/java/org/apache/oozie/tools/TestConcurrentCopyFromLocal.java 
d77eba69d 
  tools/src/test/java/org/apache/oozie/tools/TestCopyTaskCallable.java 
bce0433c6 
  tools/src/test/java/org/apache/oozie/tools/TestOozieSharelibCLI.java 
5929e5ca0 
  
tools/src/test/java/org/apache/oozie/tools/TestOozieSharelibCLIExtraArgsParser.java
 PRE-CREATION 


Diff: https://reviews.apache.org/r/67834/diff/6/

Changes: https://reviews.apache.org/r/67834/diff/5-6/


Testing
---

Tested manually + added integration test


Thanks,

Kinga Marton

[jira] [Updated] (OOZIE-2829) Improve sharelib upload to accept multiple source folders

2018-07-22 Thread Julia Kinga Marton (JIRA) 


 [ 
https://issues.apache.org/jira/browse/OOZIE-2829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Julia Kinga Marton updated OOZIE-2829:
--
Attachment: OOZIE-2829-010.patch

> Improve sharelib upload to accept multiple source folders
> -
>
> Key: OOZIE-2829
> URL: https://issues.apache.org/jira/browse/OOZIE-2829
> Project: Oozie
>  Issue Type: Bug
>  Components: tools
>Affects Versions: 5.0.0
>Reporter: Peter Cseh
>Assignee: Julia Kinga Marton
>Priority: Major
> Fix For: 5.1.0
>
> Attachments: OOZIE-2829-006.patch, OOZIE-2829-007.patch, 
> OOZIE-2829-008.patch, OOZIE-2829-009.patch, OOZIE-2829-01.patch, 
> OOZIE-2829-010.patch, OOZIE-2829-02.patch, OOZIE-2829-03.patch, 
> OOZIE-2829-04.patch, OOZIE-2829-05.patch
>
>
> Right now sharelib can be created via {{sharelib create -fs FS_URI -locallib 
> SHARED_LIBRARY}} where the SHARED_LIBRARY can be a tarbal or a folder. 
> It would be nice to have the possibility to define additional folders to be 
> uploaded into the sharelib, so the users don't have to copy or link the files 
> together on their machine.
> The syntax could be something like -additional-lib 
> sharelibName=/path/to/source/;/path/to/some/file,sharelibName2=/path/to/some/folder



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (OOZIE-3265) properties RERUN_FAIL_NODES and RERUN_SKIP_NODES should be able to appear together

2018-07-22 Thread TIAN XING (JIRA) 


 [ 
https://issues.apache.org/jira/browse/OOZIE-3265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

TIAN XING updated OOZIE-3265:
-
Attachment: OOZIE-3265-v3.patch

> properties RERUN_FAIL_NODES and RERUN_SKIP_NODES should be able to appear 
> together
> --
>
> Key: OOZIE-3265
> URL: https://issues.apache.org/jira/browse/OOZIE-3265
> Project: Oozie
>  Issue Type: Task
>Affects Versions: 5.0.0
>Reporter: TIAN XING
>Assignee: TIAN XING
>Priority: Minor
> Attachments: OOZIE-3265-v1.patch, OOZIE-3265-v2.patch, 
> OOZIE-3265-v3.patch, rerun.patch
>
>
> Currently when you re-run a workflow with property "oozie.wf.rerun.failnodes" 
>  set to true,
> you can no longer re-run it again with "oozie.wf.rerun.skip.nodes" property 
> specified, even if you set "oozie.wf.rerun.failnodes" to false.
> This kind of limitation is not reasonable. There is only one case where 
> "oozie.wf.rerun.failnodes" is true and "oozie.wf.rerun.skip.nodes" is not 
> null or empty, that should be disallowed.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (OOZIE-3265) properties RERUN_FAIL_NODES and RERUN_SKIP_NODES should be able to appear together

2018-07-22 Thread TIAN XING (JIRA) 


 [ 
https://issues.apache.org/jira/browse/OOZIE-3265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

TIAN XING updated OOZIE-3265:
-
Attachment: (was: OOZIE-3265-v3.patch)

> properties RERUN_FAIL_NODES and RERUN_SKIP_NODES should be able to appear 
> together
> --
>
> Key: OOZIE-3265
> URL: https://issues.apache.org/jira/browse/OOZIE-3265
> Project: Oozie
>  Issue Type: Task
>Affects Versions: 5.0.0
>Reporter: TIAN XING
>Assignee: TIAN XING
>Priority: Minor
> Attachments: OOZIE-3265-v1.patch, OOZIE-3265-v2.patch, rerun.patch
>
>
> Currently when you re-run a workflow with property "oozie.wf.rerun.failnodes" 
>  set to true,
> you can no longer re-run it again with "oozie.wf.rerun.skip.nodes" property 
> specified, even if you set "oozie.wf.rerun.failnodes" to false.
> This kind of limitation is not reasonable. There is only one case where 
> "oozie.wf.rerun.failnodes" is true and "oozie.wf.rerun.skip.nodes" is not 
> null or empty, that should be disallowed.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (OOZIE-2829) Improve sharelib upload to accept multiple source folders

2018-07-22 Thread Hadoop QA (JIRA) 


[ 
https://issues.apache.org/jira/browse/OOZIE-2829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16552048#comment-16552048
 ] 

Hadoop QA commented on OOZIE-2829:
--


Testing JIRA OOZIE-2829

Cleaning local git workspace



{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:green}+1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than 
132
.{color:green}+1{color} the patch adds/modifies 6 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT 
warnings
{color:green}+1 JAVADOC{color}
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc 
warning(s)
.{color:green}+1{color} the patch does not seem to introduce new Javadoc 
error(s)
.{color:red}ERROR{color}: the current HEAD has 2 Javadoc error(s)
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac 
warnings
{color:orange}0{color} There are [1] new bugs found in total that would be nice 
to have fixed.
. {color:green}+1{color} There are no new bugs found in [server].
. {color:green}+1{color} There are no new bugs found in [client].
. {color:green}+1{color} There are no new bugs found in [core].
. {color:green}+1{color} There are no new bugs found in 
[fluent-job/fluent-job-api].
. {color:green}+1{color} There are no new bugs found in [docs].
. {color:orange}0{color} There are [1] new bugs found in [tools] that would be 
nice to have fixed.
. You can find the FindBugs diff here: tools/findbugs-new.html
. {color:green}+1{color} There are no new bugs found in [examples].
. {color:green}+1{color} There are no new bugs found in [webapp].
. {color:green}+1{color} There are no new bugs found in [sharelib/streaming].
. {color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
. {color:green}+1{color} There are no new bugs found in [sharelib/distcp].
. {color:green}+1{color} There are no new bugs found in [sharelib/oozie].
. {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive2].
. {color:green}+1{color} There are no new bugs found in [sharelib/pig].
. {color:green}+1{color} There are no new bugs found in [sharelib/spark].
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA 
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:red}-1 TESTS{color}
.Tests run: 2912
.Tests failed: 1
.Tests errors: 0

.The patch failed the following testcases:

testCopyingExtraSharelibs(org.apache.oozie.tools.TestOozieSharelibCLI)

.Tests failing with errors:


{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch 


{color:red}*-1 Overall result, please check the reported -1(s)*{color}


The full output of the test-patch run is available at

. https://builds.apache.org/job/PreCommit-OOZIE-Build/680/



> Improve sharelib upload to accept multiple source folders
> -
>
> Key: OOZIE-2829
> URL: https://issues.apache.org/jira/browse/OOZIE-2829
> Project: Oozie
>  Issue Type: Bug
>  Components: tools
>Affects Versions: 5.0.0
>Reporter: Peter Cseh
>Assignee: Julia Kinga Marton
>Priority: Major
> Fix For: 5.1.0
>
> Attachments: OOZIE-2829-006.patch, OOZIE-2829-007.patch, 
> OOZIE-2829-008.patch, OOZIE-2829-009.patch, OOZIE-2829-01.patch, 
> OOZIE-2829-02.patch, OOZIE-2829-03.patch, OOZIE-2829-04.patch, 
> OOZIE-2829-05.patch
>
>
> Right now sharelib can be created via {{sharelib create -fs FS_URI -locallib 
> SHARED_LIBRARY}} where the SHARED_LIBRARY can be a tarbal or a folder. 
> It would be nice to have the possibility to define additional folders to be 
> uploaded into the sharelib, so the users don't have to copy or link the files 
> together on their machine.
> The syntax could be something like -additional-lib 
> sharelibName=/path/to/source/;/path/to/some/file,sharelibName2=/path/to/some/folder



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Failed: OOZIE-2829 PreCommit Build #680

2018-07-22 Thread Apache Jenkins Server 
Jira: https://issues.apache.org/jira/browse/OOZIE-2829
Build: https://builds.apache.org/job/PreCommit-OOZIE-Build/680/

###
## LAST 100 LINES OF THE CONSOLE 
###
[...truncated 1.75 MB...]
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [sharelib/pig].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [sharelib/spark].
[WARN] There are [1] new bugs found in total that would be nice to have fixed.
[TRACE] FindBugs diffs checked and reports created
[TRACE] Summary file size is 2581 bytes
[TRACE] Full summary file size is 1530 bytes
[TRACE] File 
[/home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/tmp/FINDBUGS_DIFF/diff/findbugs-diff-0.1.0-all.jar]
 removed
[TRACE] File 
[/home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/tmp/FINDBUGS_DIFF/diff/findbugs-diff-0.1.0-all.jar.md5sum]
 removed
  Running test-patch task BACKWARDS_COMPATIBILITY
  Running test-patch task TESTS
  Running test-patch task DISTRO


Testing JIRA OOZIE-2829

Cleaning local git workspace



+1 PATCH_APPLIES
+1 CLEAN
+1 RAW_PATCH_ANALYSIS
+1 the patch does not introduce any @author tags
+1 the patch does not introduce any tabs
+1 the patch does not introduce any trailing spaces
+1 the patch does not introduce any line longer than 132
+1 the patch adds/modifies 6 testcase(s)
+1 RAT
+1 the patch does not seem to introduce new RAT warnings
+1 JAVADOC
+1 JAVADOC
+1 the patch does not seem to introduce new Javadoc warning(s)
+1 the patch does not seem to introduce new Javadoc error(s)
ERROR: the current HEAD has 2 Javadoc error(s)
+1 COMPILE
+1 HEAD compiles
+1 patch compiles
+1 the patch does not seem to introduce new javac warnings
0 There are [1] new bugs found in total that would be nice to have fixed.
 +1 There are no new bugs found in [server].
 +1 There are no new bugs found in [client].
 +1 There are no new bugs found in [core].
 +1 There are no new bugs found in [fluent-job/fluent-job-api].
 +1 There are no new bugs found in [docs].
 0 There are [1] new bugs found in [tools] that would be nice to have fixed.
 You can find the FindBugs diff here: tools/findbugs-new.html
 +1 There are no new bugs found in [examples].
 +1 There are no new bugs found in [webapp].
 +1 There are no new bugs found in [sharelib/streaming].
 +1 There are no new bugs found in [sharelib/sqoop].
 +1 There are no new bugs found in [sharelib/distcp].
 +1 There are no new bugs found in [sharelib/oozie].
 +1 There are no new bugs found in [sharelib/hcatalog].
 +1 There are no new bugs found in [sharelib/hive].
 +1 There are no new bugs found in [sharelib/hive2].
 +1 There are no new bugs found in [sharelib/pig].
 +1 There are no new bugs found in [sharelib/spark].
+1 BACKWARDS_COMPATIBILITY
+1 the patch does not change any JPA Entity/Colum/Basic/Lob/Transient 
annotations
+1 the patch does not modify JPA files
-1 TESTS
Tests run: 2912
Tests failed: 1
Tests errors: 0

The patch failed the following testcases:

testCopyingExtraSharelibs(org.apache.oozie.tools.TestOozieSharelibCLI)

Tests failing with errors:


+1 DISTRO
+1 distro tarball builds with the patch 


-1 Overall result, please check the reported -1(s)


The full output of the test-patch run is available at

 https://builds.apache.org/job/PreCommit-OOZIE-Build/680/

Adding comment to JIRA
  % Total% Received % Xferd  Average Speed   TimeTime Time  Current
 Dload  Upload   Total   SpentLeft  Speed
  0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 
0100  34410 0  100  3441  0   5257 --:--:-- --:--:-- --:--:--  
5253{"self":"https://issues.apache.org/jira/rest/api/2/issue/13056997/comment/16552048","id":"16552048","author":{"self":"https://issues.apache.org/jira/rest/api/2/user?username=hadoopqa","name":"hadoopqa","key":"hadoopqa","avatarUrls":{"48x48":"https://issues.apache.org/jira/secure/useravatar?ownerId=hadoopqa&avatarId=10393","24x24":"https://issues.apache.org/jira/secure/useravatar?size=small&ownerId=hadoopqa&avatarId=10393","16x16":"https://issues.apache.org/jira/secure/useravatar?size=xsmall&ownerId=hadoopqa&avatarId=10393","32x32":"https://issues.apache.org/jira/secure/useravatar?size=medium&ownerId=hadoopqa&avatarId=10393"},"displayName":"Hadoop
 QA","active":true,"timeZone":"Etc/UTC"},"body":"\nTesting JIRA 
OOZIE-2829\n\nCleaning local git 
workspace\n\n\n\n{color:green}+1 
PATCH_APPLIES{color}\n{color:green}+1 CLEAN{color}\n{color:green}+1 
RAW_PATCH_ANALYSIS{color}\n.{color:green}+1{color} the patch does not 
introduce any @author tags\n.{co

[jira] [Created] (OOZIE-3308) Update oozie-setup.sh usage description in documentation

2018-07-22 Thread Julia Kinga Marton (JIRA) 
Julia Kinga Marton created OOZIE-3308:
-

 Summary: Update oozie-setup.sh usage description in documentation
 Key: OOZIE-3308
 URL: https://issues.apache.org/jira/browse/OOZIE-3308
 Project: Oozie
  Issue Type: Bug
  Components: docs
Affects Versions: 5.0.0
Reporter: Julia Kinga Marton


The documentation of the oozie-setup.sh -create contains the following 
information:

"(action fails if sharelib is already installed in HDFS)", which is not true 
anymore. This information should be removed.

[https://oozie.apache.org/docs/5.0.0/AG_Install.html]

[https://github.com/apache/oozie/blob/master/docs/src/site/twiki/AG_Install.twiki#L61-L62]

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (OOZIE-2829) Improve sharelib upload to accept multiple source folders

2018-07-22 Thread Hadoop QA (JIRA) 


[ 
https://issues.apache.org/jira/browse/OOZIE-2829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16552015#comment-16552015
 ] 

Hadoop QA commented on OOZIE-2829:
--

PreCommit-OOZIE-Build started


> Improve sharelib upload to accept multiple source folders
> -
>
> Key: OOZIE-2829
> URL: https://issues.apache.org/jira/browse/OOZIE-2829
> Project: Oozie
>  Issue Type: Bug
>  Components: tools
>Affects Versions: 5.0.0
>Reporter: Peter Cseh
>Assignee: Julia Kinga Marton
>Priority: Major
> Fix For: 5.1.0
>
> Attachments: OOZIE-2829-006.patch, OOZIE-2829-007.patch, 
> OOZIE-2829-008.patch, OOZIE-2829-009.patch, OOZIE-2829-01.patch, 
> OOZIE-2829-02.patch, OOZIE-2829-03.patch, OOZIE-2829-04.patch, 
> OOZIE-2829-05.patch
>
>
> Right now sharelib can be created via {{sharelib create -fs FS_URI -locallib 
> SHARED_LIBRARY}} where the SHARED_LIBRARY can be a tarbal or a folder. 
> It would be nice to have the possibility to define additional folders to be 
> uploaded into the sharelib, so the users don't have to copy or link the files 
> together on their machine.
> The syntax could be something like -additional-lib 
> sharelibName=/path/to/source/;/path/to/some/file,sharelibName2=/path/to/some/folder



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Failed: OOZIE-3265 PreCommit Build #679

2018-07-22 Thread Apache Jenkins Server 
Jira: https://issues.apache.org/jira/browse/OOZIE-3265
Build: https://builds.apache.org/job/PreCommit-OOZIE-Build/679/

###
## LAST 100 LINES OF THE CONSOLE 
###
[...truncated 31.16 KB...]
at 
hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1741)
at 
hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357)
at hudson.remoting.Channel.call(Channel.java:955)
at 
org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.execute(RemoteGitImpl.java:146)
at sun.reflect.GeneratedMethodAccessor1143.invoke(Unknown 
Source)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.invoke(RemoteGitImpl.java:132)
at com.sun.proxy.$Proxy118.execute(Unknown Source)
at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:886)
at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1155)
at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1186)
at hudson.scm.SCM.checkout(SCM.java:504)
at 
hudson.model.AbstractProject.checkout(AbstractProject.java:1208)
at 
hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574)
at 
jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
at 
hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
at hudson.model.Run.execute(Run.java:1794)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at 
hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:429)
ERROR: Error fetching remote repo 'origin'
Retrying after 10 seconds
 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url 
 > https://git-wip-us.apache.org/repos/asf/oozie.git # timeout=10
Fetching upstream changes from https://git-wip-us.apache.org/repos/asf/oozie.git
 > git --version # timeout=10
 > git fetch --tags --progress 
 > https://git-wip-us.apache.org/repos/asf/oozie.git 
 > +refs/heads/*:refs/remotes/origin/*
ERROR: Error fetching remote repo 'origin'
hudson.plugins.git.GitException: Failed to fetch from 
https://git-wip-us.apache.org/repos/asf/oozie.git
at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:888)
at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1155)
at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1186)
at hudson.scm.SCM.checkout(SCM.java:504)
at hudson.model.AbstractProject.checkout(AbstractProject.java:1208)
at 
hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574)
at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
at 
hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
at hudson.model.Run.execute(Run.java:1794)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:429)
Caused by: hudson.plugins.git.GitException: Command "git fetch --tags 
--progress https://git-wip-us.apache.org/repos/asf/oozie.git 
+refs/heads/*:refs/remotes/origin/*" returned status code 128:
stdout: 
stderr: remote: Counting objects: 14689, done.
remote: Compressing objects:   0% (1/5983)   remote: Compressing 
objects:   1% (60/5983)   remote: Compressing objects:   2% (120/5983)  
 remote: Compressing objects:   3% (180/5983)   remote: 
Compressing objects:   4% (240/5983)   remote: Compressing objects:   
5% (300/5983)   remote: Compressing objects:   6% (359/5983)   
remote: Compressing objects:   7% (419/5983)   remote: Compressing 
objects:   8% (479/5983)   remote: Compressing objects:   9% (539/5983) 
  remote: Compressing objects:  10% (599/5983)   remote: 
Compressing objects:  11% (659/5983)   remote: Compressing objects:  
12% (718/5983)   remote: Compressing objects:  13% (778/5983)   
remote: Compressing objects:  14% (838/5983)   remote: Compressing 
objects:  15% (898/5983)   remote: Compressing objects:  16% (958/5983) 
  remote: Compressing objects:  17% (1018/5983)   remote: 
Compressing objects:  18% (1077/5983)   remote: Compressing objects:  
19% (1137/5983)   remote: Compressing objects:  20% (11

[jira] [Updated] (OOZIE-3265) properties RERUN_FAIL_NODES and RERUN_SKIP_NODES should be able to appear together

2018-07-22 Thread TIAN XING (JIRA) 


 [ 
https://issues.apache.org/jira/browse/OOZIE-3265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

TIAN XING updated OOZIE-3265:
-
Attachment: OOZIE-3265-v3.patch

> properties RERUN_FAIL_NODES and RERUN_SKIP_NODES should be able to appear 
> together
> --
>
> Key: OOZIE-3265
> URL: https://issues.apache.org/jira/browse/OOZIE-3265
> Project: Oozie
>  Issue Type: Task
>Affects Versions: 5.0.0
>Reporter: TIAN XING
>Assignee: TIAN XING
>Priority: Minor
> Attachments: OOZIE-3265-v1.patch, OOZIE-3265-v2.patch, 
> OOZIE-3265-v3.patch, rerun.patch
>
>
> Currently when you re-run a workflow with property "oozie.wf.rerun.failnodes" 
>  set to true,
> you can no longer re-run it again with "oozie.wf.rerun.skip.nodes" property 
> specified, even if you set "oozie.wf.rerun.failnodes" to false.
> This kind of limitation is not reasonable. There is only one case where 
> "oozie.wf.rerun.failnodes" is true and "oozie.wf.rerun.skip.nodes" is not 
> null or empty, that should be disallowed.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Review Request 67834: OOZIE-2829 Improve sharelib upload to accept multiple source folders

2018-07-22 Thread Kinga Marton via Review Board 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67834/
---

(Updated July 22, 2018, 1:22 p.m.)


Review request for oozie, András Piros, Peter Cseh, and Peter Bacsko.


Changes
---

- fixed review comments
- added some additional tests
- changed the sharelibs separator to '+'. However I am not completely satisfied 
with this solution, but I don't have any better ideas.


Repository: oozie-git


Description
---

Right now sharelib can be created via sharelib create -fs FS_URI -locallib 
SHARED_LIBRARY where the SHARED_LIBRARY can be a tarbal or a folder.
It would be nice to have the possibility to define additional folders to be 
uploaded into the sharelib, so the users don't have to copy or link the files 
together on their machine.
The syntax could be something like -additional-lib 
sharelibName=/path/to/source/;/path/to/some/file,sharelibName2=/path/to/some/folder


Diffs (updated)
-

  docs/src/site/twiki/AG_Install.twiki 46363a3b3 
  tools/src/main/java/org/apache/oozie/tools/OozieSharelibCLI.java 75e932c02 
  
tools/src/test/java/org/apache/oozie/tools/IntegrationTestOozieSharelibCLI.java 
PRE-CREATION 
  tools/src/test/java/org/apache/oozie/tools/OozieSharelibFileOperations.java 
d344300ed 
  tools/src/test/java/org/apache/oozie/tools/TestConcurrentCopyFromLocal.java 
d77eba69d 
  tools/src/test/java/org/apache/oozie/tools/TestCopyTaskCallable.java 
bce0433c6 
  tools/src/test/java/org/apache/oozie/tools/TestOozieSharelibCLI.java 
5929e5ca0 
  
tools/src/test/java/org/apache/oozie/tools/TestOozieSharelibCLIExtraArgsParser.java
 PRE-CREATION 


Diff: https://reviews.apache.org/r/67834/diff/5/

Changes: https://reviews.apache.org/r/67834/diff/4-5/


Testing
---

Tested manually + added integration test


Thanks,

Kinga Marton

[jira] [Updated] (OOZIE-2829) Improve sharelib upload to accept multiple source folders

2018-07-22 Thread Julia Kinga Marton (JIRA) 


 [ 
https://issues.apache.org/jira/browse/OOZIE-2829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Julia Kinga Marton updated OOZIE-2829:
--
Attachment: OOZIE-2829-009.patch

> Improve sharelib upload to accept multiple source folders
> -
>
> Key: OOZIE-2829
> URL: https://issues.apache.org/jira/browse/OOZIE-2829
> Project: Oozie
>  Issue Type: Bug
>  Components: tools
>Affects Versions: 5.0.0
>Reporter: Peter Cseh
>Assignee: Julia Kinga Marton
>Priority: Major
> Fix For: 5.1.0
>
> Attachments: OOZIE-2829-006.patch, OOZIE-2829-007.patch, 
> OOZIE-2829-008.patch, OOZIE-2829-009.patch, OOZIE-2829-01.patch, 
> OOZIE-2829-02.patch, OOZIE-2829-03.patch, OOZIE-2829-04.patch, 
> OOZIE-2829-05.patch
>
>
> Right now sharelib can be created via {{sharelib create -fs FS_URI -locallib 
> SHARED_LIBRARY}} where the SHARED_LIBRARY can be a tarbal or a folder. 
> It would be nice to have the possibility to define additional folders to be 
> uploaded into the sharelib, so the users don't have to copy or link the files 
> together on their machine.
> The syntax could be something like -additional-lib 
> sharelibName=/path/to/source/;/path/to/some/file,sharelibName2=/path/to/some/folder



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (OOZIE-2942) [examples] Fix Findbugs warnings

2018-07-22 Thread Julia Kinga Marton (JIRA) 


[ 
https://issues.apache.org/jira/browse/OOZIE-2942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16552011#comment-16552011
 ] 

Julia Kinga Marton commented on OOZIE-2942:
---

As I cannot modify the original review, I have created a new Review Board 
review: [https://reviews.apache.org/r/68008/]

 

> [examples] Fix Findbugs warnings
> 
>
> Key: OOZIE-2942
> URL: https://issues.apache.org/jira/browse/OOZIE-2942
> Project: Oozie
>  Issue Type: Sub-task
>  Components: build
>Reporter: Jan Hentschel
>Assignee: Julia Kinga Marton
>Priority: Major
> Fix For: 5.1.0
>
> Attachments: OOZIE-2942-003.patch, OOZIE-2942-004.patch, 
> OOZIE-2942-1.patch, OOZIE-2942-2.patch, OOZIE-2942.005.patch, 
> OOZIE-2942.006.patch
>
>
> Currently Findbugs complains about the following warnings in the 
> *oozie-examples* module:
> {code}
> [INFO] org.apache.oozie.example.DateList.main(String[]) may fail to close 
> stream on exception [org.apache.oozie.example.DateList] At 
> DateList.java:[line 69]
> [INFO] Nullcheck of date at line 55 of value previously dereferenced in 
> org.apache.oozie.example.DateList.main(String[]) 
> [org.apache.oozie.example.DateList, org.apache.oozie.example.DateList] At 
> DateList.java:[line 55]Redundant null check at DateList.java:[line 62]
> [INFO] Private method 
> org.apache.oozie.example.DateList.formatDateUTC(Calendar) is never called 
> [org.apache.oozie.example.DateList] At DateList.java:[line 97]
> [INFO] org.apache.oozie.example.LocalOozieExample.execute(String[]) may fail 
> to clean up java.io.InputStream [org.apache.oozie.example.LocalOozieExample, 
> org.apache.oozie.example.LocalOozieExample, 
> org.apache.oozie.example.LocalOozieExample, 
> org.apache.oozie.example.LocalOozieExample, 
> org.apache.oozie.example.LocalOozieExample, 
> org.apache.oozie.example.LocalOozieExample, 
> org.apache.oozie.example.LocalOozieExample, 
> org.apache.oozie.example.LocalOozieExample, 
> org.apache.oozie.example.LocalOozieExample] Obligation to clean up resource 
> created at LocalOozieExample.java:[line 72] is not dischargedPath continues 
> at LocalOozieExample.java:[line 76]Path continues at 
> LocalOozieExample.java:[line 77]Path continues at 
> LocalOozieExample.java:[line 78]Path continues at 
> LocalOozieExample.java:[line 81]Path continues at 
> LocalOozieExample.java:[line 88]Path continues at 
> LocalOozieExample.java:[line 89]Path continues at 
> LocalOozieExample.java:[line 91]Path continues at 
> LocalOozieExample.java:[line 100]
> [INFO] org.apache.oozie.example.LocalOozieExample.execute(String[]) may fail 
> to close stream [org.apache.oozie.example.LocalOozieExample] At 
> LocalOozieExample.java:[line 72]
> [INFO] org.apache.oozie.example.Repeatable.getBaseline() may expose internal 
> representation by returning Repeatable.baseline 
> [org.apache.oozie.example.Repeatable] At Repeatable.java:[line 168]
> [INFO] org.apache.oozie.example.Repeatable.setBaseline(Date) may expose 
> internal representation by storing an externally mutable object into 
> Repeatable.baseline [org.apache.oozie.example.Repeatable] At 
> Repeatable.java:[line 172]
> {code}
> They should be fixed to get the code more reliable.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Review Request 68008: OOZIE-2942 [examples] Fix Findbugs warnings

2018-07-22 Thread Kinga Marton via Review Board 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68008/
---

Review request for oozie, András Piros and Peter Cseh.


Repository: oozie-git


Description
---

Currently Findbugs complains about the following warnings in the oozie-examples 
module:
```
[INFO] org.apache.oozie.example.DateList.main(String[]) may fail to close 
stream on exception [org.apache.oozie.example.DateList] At DateList.java:[line 
69]
[INFO] Nullcheck of date at line 55 of value previously dereferenced in 
org.apache.oozie.example.DateList.main(String[]) 
[org.apache.oozie.example.DateList, org.apache.oozie.example.DateList] At 
DateList.java:[line 55]Redundant null check at DateList.java:[line 62]
[INFO] Private method org.apache.oozie.example.DateList.formatDateUTC(Calendar) 
is never called [org.apache.oozie.example.DateList] At DateList.java:[line 97]
[INFO] org.apache.oozie.example.LocalOozieExample.execute(String[]) may fail to 
clean up java.io.InputStream [org.apache.oozie.example.LocalOozieExample, 
org.apache.oozie.example.LocalOozieExample, 
org.apache.oozie.example.LocalOozieExample, 
org.apache.oozie.example.LocalOozieExample, 
org.apache.oozie.example.LocalOozieExample, 
org.apache.oozie.example.LocalOozieExample, 
org.apache.oozie.example.LocalOozieExample, 
org.apache.oozie.example.LocalOozieExample, 
org.apache.oozie.example.LocalOozieExample] Obligation to clean up resource 
created at LocalOozieExample.java:[line 72] is not dischargedPath continues at 
LocalOozieExample.java:[line 76]Path continues at LocalOozieExample.java:[line 
77]Path continues at LocalOozieExample.java:[line 78]Path continues at 
LocalOozieExample.java:[line 81]Path continues at LocalOozieExample.java:[line 
88]Path continues at LocalOozieExample.java:[line 89]Path continues at 
LocalOozieExample.java:[line 91]Path continues at LocalOozieExample.java:[line 
10
 0]
[INFO] org.apache.oozie.example.LocalOozieExample.execute(String[]) may fail to 
close stream [org.apache.oozie.example.LocalOozieExample] At 
LocalOozieExample.java:[line 72]
[INFO] org.apache.oozie.example.Repeatable.getBaseline() may expose internal 
representation by returning Repeatable.baseline 
[org.apache.oozie.example.Repeatable] At Repeatable.java:[line 168]
[INFO] org.apache.oozie.example.Repeatable.setBaseline(Date) may expose 
internal representation by storing an externally mutable object into 
Repeatable.baseline [org.apache.oozie.example.Repeatable] At 
Repeatable.java:[line 172]
```

They should be fixed to get the code more reliable.


Diffs
-

  examples/src/main/java/org/apache/oozie/example/DateList.java 7e574cbe 
  examples/src/main/java/org/apache/oozie/example/LocalOozieExample.java 
c9f5697c 
  examples/src/main/java/org/apache/oozie/example/Repeatable.java ee863251 


Diff: https://reviews.apache.org/r/68008/diff/1/


Testing
---


Thanks,

Kinga Marton

Re: Review Request 67834: OOZIE-2829 Improve sharelib upload to accept multiple source folders

2018-07-22 Thread Kinga Marton via Review Board 


> On July 6, 2018, 1:40 p.m., András Piros wrote:
> > tools/src/main/java/org/apache/oozie/tools/OozieSharelibCLI.java
> > Lines 88 (patched)
> > 
> >
> > Problem is we can have ```hdfs://my/jar.jar#myjar.jar``` inside 
> > sharelib paths.
> > 
> > Best is to go with e.g. `;` that cannot be present as HDFS or local 
> > file.
> 
> Kinga Marton wrote:
> Using semicolon(;) is not a good ideea because the shell handles it as a 
> command separator. What about using & ?
> 
> András Piros wrote:
> I'm OK using colon `,` as the separator between sharelib name-path pairs, 
> and pipe `|` between sharelib paths like this:
> ```
> additional-lib 
> sharelibName1=/path/to/source/|/path/to/some/file,sharelibName2=/path/to/some/folder
> ```
> 
> Kinga Marton wrote:
> The pipe will not work, since it has the same problem as the semicolon. 
> It is interpreted by the Shell. It can work only if is surrounded by double 
> quotes. The problem is that every character that seems to be logic as a 
> delimiter, is interpreted by the Shell. 
> What about using '+' as sharelibs delimiter?
> Using '+' would result in the following command:
> ```
> -extralibs 
> sharelibName=/path/to/source/,/path/to/some/file+sharelibName2=/path/to/some/folder,hdfs://my/jar.jar#myjar.jar
> ```
> 
> Or another solution would be to use double quotes if we want to specify 
> more than one extra sharelib. This way we can even use the semicolon (;)

I have changes the ":" to "+", since I dont't have any better idea. However I 
am not 100% satisfied with this solution.


- Kinga


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67834/#review205796
---


On July 6, 2018, 8:36 a.m., Kinga Marton wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/67834/
> ---
> 
> (Updated July 6, 2018, 8:36 a.m.)
> 
> 
> Review request for oozie, András Piros, Peter Cseh, and Peter Bacsko.
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> Right now sharelib can be created via sharelib create -fs FS_URI -locallib 
> SHARED_LIBRARY where the SHARED_LIBRARY can be a tarbal or a folder.
> It would be nice to have the possibility to define additional folders to be 
> uploaded into the sharelib, so the users don't have to copy or link the files 
> together on their machine.
> The syntax could be something like -additional-lib 
> sharelibName=/path/to/source/;/path/to/some/file,sharelibName2=/path/to/some/folder
> 
> 
> Diffs
> -
> 
>   docs/src/site/twiki/AG_Install.twiki 46363a3b3 
>   tools/src/main/java/org/apache/oozie/tools/OozieSharelibCLI.java 75e932c02 
>   tools/src/test/java/org/apache/oozie/tools/OozieSharelibFileOperations.java 
> d344300ed 
>   tools/src/test/java/org/apache/oozie/tools/TestConcurrentCopyFromLocal.java 
> d77eba69d 
>   tools/src/test/java/org/apache/oozie/tools/TestCopyTaskCallable.java 
> bce0433c6 
>   tools/src/test/java/org/apache/oozie/tools/TestOozieSharelibCLI.java 
> 5929e5ca0 
>   
> tools/src/test/java/org/apache/oozie/tools/diag/IntegrationTestOozieSharelibCLI.java
>  PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/67834/diff/4/
> 
> 
> Testing
> ---
> 
> Tested manually + added integration test
> 
> 
> Thanks,
> 
> Kinga Marton
> 
>