[jira] [Updated] (OOZIE-3321) PySpark example fails

[Daniel Becker (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-3321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Daniel Becker updated OOZIE-3321:
-
Attachment: OOZIE-3321.3.patch

> PySpark example fails
> -
>
> Key: OOZIE-3321
> URL: https://issues.apache.org/jira/browse/OOZIE-3321
> Project: Oozie
>  Issue Type: Bug
>  Components: examples
>Affects Versions: 5.0.0, 5.1.0
>Reporter: Daniel Becker
>Assignee: Daniel Becker
>Priority: Minor
> Attachments: OOZIE-3321.1.patch, OOZIE-3321.2.patch, 
> OOZIE-3321.3.patch
>
>
> The PySpark example fails with the following exception:
> {noformat}
> ACTION[001-180806145017687-oozie-oozi-W@spark-node] Launcher exception: 
> Missing py4j and/or pyspark zip files. Please add them to the lib folder or 
> to the Spark sharelib.
> org.apache.oozie.action.hadoop.OozieActionConfiguratorException: Missing py4j 
> and/or pyspark zip files. Please add them to the lib folder or to the Spark 
> sharelib.
>   at 
> org.apache.oozie.action.hadoop.SparkMain.getMatchingPyFile(SparkMain.java:151)
>   at 
> org.apache.oozie.action.hadoop.SparkMain.createPySparkLibFolder(SparkMain.java:132)
>   at org.apache.oozie.action.hadoop.SparkMain.run(SparkMain.java:77)
>   at 
> org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:101)
>   at org.apache.oozie.action.hadoop.SparkMain.main(SparkMain.java:60)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:498)
>   at 
> org.apache.oozie.action.hadoop.LauncherAM.runActionMain(LauncherAM.java:410)
>   at 
> org.apache.oozie.action.hadoop.LauncherAM.access$300(LauncherAM.java:55)
>   at org.apache.oozie.action.hadoop.LauncherAM$2.run(LauncherAM.java:223)
>   at java.security.AccessController.doPrivileged(Native Method)
>   at javax.security.auth.Subject.doAs(Subject.java:422)
>   at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1692)
>   at org.apache.oozie.action.hadoop.LauncherAM.run(LauncherAM.java:217)
>   at org.apache.oozie.action.hadoop.LauncherAM$1.run(LauncherAM.java:153)
>   at java.security.AccessController.doPrivileged(Native Method)
>   at javax.security.auth.Subject.doAs(Subject.java:422)
>   at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1692)
>   at org.apache.oozie.action.hadoop.LauncherAM.main(LauncherAM.java:141)
> {noformat}
> This is because py4j-0.9-src.zip and pyspark.zip are not available. They are 
> needed for the example to run, so we should either add them to the sharelib 
> or if they are not important enough to be in the sharelib, we should add them 
> to the example's lib directory. Currently these two files can be found in the 
> codebase under sharelib/spark/src/test/resources. We should also decide what 
> to do with these files there, to avoid duplication of resource files.
> If I add the files manually to either the sharelib or the lib directory of 
> the example, the example runs successfilly.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (OOZIE-3321) PySpark example fails

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-3321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16572824#comment-16572824
 ] 

Hadoop QA commented on OOZIE-3321:
--

PreCommit-OOZIE-Build started


> PySpark example fails
> -
>
> Key: OOZIE-3321
> URL: https://issues.apache.org/jira/browse/OOZIE-3321
> Project: Oozie
>  Issue Type: Bug
>  Components: examples
>Affects Versions: 5.0.0, 5.1.0
>Reporter: Daniel Becker
>Assignee: Daniel Becker
>Priority: Minor
> Attachments: OOZIE-3321.1.patch, OOZIE-3321.2.patch, 
> OOZIE-3321.3.patch
>
>
> The PySpark example fails with the following exception:
> {noformat}
> ACTION[001-180806145017687-oozie-oozi-W@spark-node] Launcher exception: 
> Missing py4j and/or pyspark zip files. Please add them to the lib folder or 
> to the Spark sharelib.
> org.apache.oozie.action.hadoop.OozieActionConfiguratorException: Missing py4j 
> and/or pyspark zip files. Please add them to the lib folder or to the Spark 
> sharelib.
>   at 
> org.apache.oozie.action.hadoop.SparkMain.getMatchingPyFile(SparkMain.java:151)
>   at 
> org.apache.oozie.action.hadoop.SparkMain.createPySparkLibFolder(SparkMain.java:132)
>   at org.apache.oozie.action.hadoop.SparkMain.run(SparkMain.java:77)
>   at 
> org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:101)
>   at org.apache.oozie.action.hadoop.SparkMain.main(SparkMain.java:60)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:498)
>   at 
> org.apache.oozie.action.hadoop.LauncherAM.runActionMain(LauncherAM.java:410)
>   at 
> org.apache.oozie.action.hadoop.LauncherAM.access$300(LauncherAM.java:55)
>   at org.apache.oozie.action.hadoop.LauncherAM$2.run(LauncherAM.java:223)
>   at java.security.AccessController.doPrivileged(Native Method)
>   at javax.security.auth.Subject.doAs(Subject.java:422)
>   at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1692)
>   at org.apache.oozie.action.hadoop.LauncherAM.run(LauncherAM.java:217)
>   at org.apache.oozie.action.hadoop.LauncherAM$1.run(LauncherAM.java:153)
>   at java.security.AccessController.doPrivileged(Native Method)
>   at javax.security.auth.Subject.doAs(Subject.java:422)
>   at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1692)
>   at org.apache.oozie.action.hadoop.LauncherAM.main(LauncherAM.java:141)
> {noformat}
> This is because py4j-0.9-src.zip and pyspark.zip are not available. They are 
> needed for the example to run, so we should either add them to the sharelib 
> or if they are not important enough to be in the sharelib, we should add them 
> to the example's lib directory. Currently these two files can be found in the 
> codebase under sharelib/spark/src/test/resources. We should also decide what 
> to do with these files there, to avoid duplication of resource files.
> If I add the files manually to either the sharelib or the lib directory of 
> the example, the example runs successfilly.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (OOZIE-3314) Remove findbugs-filter.xml and convert its contents to annotations

[Andras Salamon (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-3314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andras Salamon updated OOZIE-3314:
--
Attachment: OOZIE-3314-2.patch

> Remove findbugs-filter.xml and convert its contents to annotations
> --
>
> Key: OOZIE-3314
> URL: https://issues.apache.org/jira/browse/OOZIE-3314
> Project: Oozie
>  Issue Type: Bug
>  Components: core
>Reporter: Peter Bacsko
>Assignee: Andras Salamon
>Priority: Major
> Attachments: OOZIE-3314-1.patch, OOZIE-3314-2.patch
>
>
> In oozie-core, we have a file called "findbugs-filter.xml" which tells 
> findbugs that it shluld ignore certain problems in a couple of classes.
> However if we try to compile a sub-module or run findbugs directly (let's say 
> in a sharelib project), the build will fail because it won't be able to open 
> findbugs.xml. It's not straightforward how to define the path of the XML in a 
> way that it makes this xml accessible regardless of what module you're 
> compiling.
> It's better to just convert its contents to annotations - especially since we 
> already use this method too.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (OOZIE-3314) Remove findbugs-filter.xml and convert its contents to annotations

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-3314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16572908#comment-16572908
 ] 

Hadoop QA commented on OOZIE-3314:
--

PreCommit-OOZIE-Build started


> Remove findbugs-filter.xml and convert its contents to annotations
> --
>
> Key: OOZIE-3314
> URL: https://issues.apache.org/jira/browse/OOZIE-3314
> Project: Oozie
>  Issue Type: Bug
>  Components: core
>Reporter: Peter Bacsko
>Assignee: Andras Salamon
>Priority: Major
> Attachments: OOZIE-3314-1.patch, OOZIE-3314-2.patch
>
>
> In oozie-core, we have a file called "findbugs-filter.xml" which tells 
> findbugs that it shluld ignore certain problems in a couple of classes.
> However if we try to compile a sub-module or run findbugs directly (let's say 
> in a sharelib project), the build will fail because it won't be able to open 
> findbugs.xml. It's not straightforward how to define the path of the XML in a 
> way that it makes this xml accessible regardless of what module you're 
> compiling.
> It's better to just convert its contents to annotations - especially since we 
> already use this method too.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Success: OOZIE-3321 PreCommit Build #736

[Apache Jenkins Server]

Jira: https://issues.apache.org/jira/browse/OOZIE-3321
Build: https://builds.apache.org/job/PreCommit-OOZIE-Build/736/

###
## LAST 100 LINES OF THE CONSOLE 
###
[...truncated 1.75 MB...]
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [sharelib/hcatalog].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [sharelib/sqoop].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [sharelib/oozie].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [sharelib/distcp].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [sharelib/spark].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [client].
[INFO] There are no new bugs found totally].
[TRACE] FindBugs diffs checked and reports created
[TRACE] Summary file size is 2456 bytes
[TRACE] Full summary file size is 1399 bytes
[TRACE] File 
[/home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/tmp/FINDBUGS_DIFF/diff/findbugs-diff-0.1.0-all.jar]
 removed
[TRACE] File 
[/home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/tmp/FINDBUGS_DIFF/diff/findbugs-diff-0.1.0-all.jar.md5sum]
 removed
  Running test-patch task BACKWARDS_COMPATIBILITY
  Running test-patch task TESTS
  Running test-patch task DISTRO


Testing JIRA OOZIE-3321

Cleaning local git workspace



+1 PATCH_APPLIES
+1 CLEAN
+1 RAW_PATCH_ANALYSIS
+1 the patch does not introduce any @author tags
+1 the patch does not introduce any tabs
+1 the patch does not introduce any trailing spaces
+1 the patch does not introduce any line longer than 132
+1 the patch adds/modifies 1 testcase(s)
+1 RAT
+1 the patch does not seem to introduce new RAT warnings
+1 JAVADOC
+1 JAVADOC
+1 the patch does not seem to introduce new Javadoc warning(s)
WARNING: the current HEAD has 100 Javadoc warning(s)
+1 the patch does not seem to introduce new Javadoc error(s)
+1 COMPILE
+1 HEAD compiles
+1 patch compiles
+1 the patch does not seem to introduce new javac warnings
+1 There are no new bugs found in total.
 +1 There are no new bugs found in [examples].
 +1 There are no new bugs found in [webapp].
 +1 There are no new bugs found in [core].
 +1 There are no new bugs found in [tools].
 +1 There are no new bugs found in [fluent-job/fluent-job-api].
 +1 There are no new bugs found in [server].
 +1 There are no new bugs found in [docs].
 +1 There are no new bugs found in [sharelib/hive2].
 +1 There are no new bugs found in [sharelib/pig].
 +1 There are no new bugs found in [sharelib/streaming].
 +1 There are no new bugs found in [sharelib/hive].
 +1 There are no new bugs found in [sharelib/hcatalog].
 +1 There are no new bugs found in [sharelib/sqoop].
 +1 There are no new bugs found in [sharelib/oozie].
 +1 There are no new bugs found in [sharelib/distcp].
 +1 There are no new bugs found in [sharelib/spark].
 +1 There are no new bugs found in [client].
+1 BACKWARDS_COMPATIBILITY
+1 the patch does not change any JPA Entity/Colum/Basic/Lob/Transient 
annotations
+1 the patch does not modify JPA files
+1 TESTS
Tests run: 2914
Tests failed at first run:
TestCoordActionInputCheckXCommandNonUTC>TestCoordActionInputCheckXCommand#testNone
For the complete list of flaky tests, see TEST-SUMMARY-FULL files.
+1 DISTRO
+1 distro tarball builds with the patch 


+1 Overall result, good!, no -1s

 There is at least one warning, please check

The full output of the test-patch run is available at

 https://builds.apache.org/job/PreCommit-OOZIE-Build/736/

Adding comment to JIRA
  % Total% Received % Xferd  Average Speed   TimeTime Time  Current
 Dload  Upload   Total   SpentLeft  Speed
  0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 0  
0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 
0{"self":"https://issues.apache.org/jira/rest/api/2/issue/13177130/comment/16572910","id":"16572910","author":{"self":"https://issues.apache.org/jira/rest/api/2/user?username=hadoopqa","name":"hadoopqa","key":"hadoopqa","avatarUrls":{"48x48":"https://issues.apache.org/jira/secure/useravatar?ownerId=hadoopqa&avatarId=10393","24x24":"https://issues.apache.org/jira/secure/useravatar?size=small&ownerId=hadoopqa&avatarId=10393","16x16":"https://issues.apache.org/jira/secure/useravatar?size=xsmall&ownerId=hadoopqa&avatarId=10393","32x32":"https://issues.apache.org/jira/secure/userava

[jira] [Commented] (OOZIE-3321) PySpark example fails

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-3321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16572910#comment-16572910
 ] 

Hadoop QA commented on OOZIE-3321:
--


Testing JIRA OOZIE-3321

Cleaning local git workspace



{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:green}+1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than 
132
.{color:green}+1{color} the patch adds/modifies 1 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT 
warnings
{color:green}+1 JAVADOC{color}
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc 
warning(s)
.{color:red}WARNING{color}: the current HEAD has 100 Javadoc warning(s)
.{color:green}+1{color} the patch does not seem to introduce new Javadoc 
error(s)
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac 
warnings
{color:green}+1{color} There are no new bugs found in total.
. {color:green}+1{color} There are no new bugs found in [examples].
. {color:green}+1{color} There are no new bugs found in [webapp].
. {color:green}+1{color} There are no new bugs found in [core].
. {color:green}+1{color} There are no new bugs found in [tools].
. {color:green}+1{color} There are no new bugs found in 
[fluent-job/fluent-job-api].
. {color:green}+1{color} There are no new bugs found in [server].
. {color:green}+1{color} There are no new bugs found in [docs].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive2].
. {color:green}+1{color} There are no new bugs found in [sharelib/pig].
. {color:green}+1{color} There are no new bugs found in [sharelib/streaming].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive].
. {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
. {color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
. {color:green}+1{color} There are no new bugs found in [sharelib/oozie].
. {color:green}+1{color} There are no new bugs found in [sharelib/distcp].
. {color:green}+1{color} There are no new bugs found in [sharelib/spark].
. {color:green}+1{color} There are no new bugs found in [client].
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA 
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:green}+1 TESTS{color}
.Tests run: 2914
.{color:orange}Tests failed at first run:{color}
TestCoordActionInputCheckXCommandNonUTC>TestCoordActionInputCheckXCommand#testNone
.For the complete list of flaky tests, see TEST-SUMMARY-FULL files.
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch 


{color:green}*+1 Overall result, good!, no -1s*{color}

{color:red}. There is at least one warning, please check{color}

The full output of the test-patch run is available at

. https://builds.apache.org/job/PreCommit-OOZIE-Build/736/



> PySpark example fails
> -
>
> Key: OOZIE-3321
> URL: https://issues.apache.org/jira/browse/OOZIE-3321
> Project: Oozie
>  Issue Type: Bug
>  Components: examples
>Affects Versions: 5.0.0, 5.1.0
>Reporter: Daniel Becker
>Assignee: Daniel Becker
>Priority: Minor
> Attachments: OOZIE-3321.1.patch, OOZIE-3321.2.patch, 
> OOZIE-3321.3.patch
>
>
> The PySpark example fails with the following exception:
> {noformat}
> ACTION[001-180806145017687-oozie-oozi-W@spark-node] Launcher exception: 
> Missing py4j and/or pyspark zip files. Please add them to the lib folder or 
> to the Spark sharelib.
> org.apache.oozie.action.hadoop.OozieActionConfiguratorException: Missing py4j 
> and/or pyspark zip files. Please add them to the lib folder or to the Spark 
> sharelib.
>   at 
> org.apache.oozie.action.hadoop.SparkMain.getMatchingPyFile(SparkMain.java:151)
>   at 
> org.apache.oozie.action.hadoop.SparkMain.createPySparkLibFolder(SparkMain.java:132)
>   at org.apache.oozie.action.hadoop.SparkMain.run(SparkMain.java:77)
>   at 
> org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:101)
>   at org.apache.oozie.action.hadoop.SparkMain.main(SparkMain.java:60)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.in

Re: Review Request 68255: OOZIE 3314 - Remove findbugs-filter.xml and convert its contents to annotations

[Andras Salamon]

> On Aug. 7, 2018, 1:01 p.m., András Piros wrote:
> > core/src/main/java/org/apache/oozie/command/XCommand.java
> > Lines 84 (patched)
> > 
> >
> > Typo: eventService

Fixed it, later removed the annotation.


> On Aug. 7, 2018, 1:01 p.m., András Piros wrote:
> > core/src/main/java/org/apache/oozie/command/XCommand.java
> > Line 82 (original), 85 (patched)
> > 
> >
> > Actually `eventService` is only written from inside one of the 
> > constructors, so it could - and should - be `static final`. Maybe can also 
> > be `private` and provide a getter for the ones wanting to reach it, or at 
> > least a `@VisibleForTesting`, when that's the case.

Writing the static variable from the constuctor is not the best idea, findbugs 
also gives us ST_WRITE_TO_STATIC_FROM_INSTANCE_METHOD warning. I've replaced 
the variable with a protected static getEventService method.


> On Aug. 7, 2018, 1:01 p.m., András Piros wrote:
> > core/src/main/java/org/apache/oozie/util/Instrumentation.java
> > Lines 710-711 (patched)
> > 
> >
> > Any other way to refactor?

Refactored. Annotation eliminated.


- Andras


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68255/#review206941
---


On Aug. 7, 2018, 12:54 p.m., Andras Salamon wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/68255/
> ---
> 
> (Updated Aug. 7, 2018, 12:54 p.m.)
> 
> 
> Review request for oozie, András Piros and Kinga Marton.
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> 3 excptions replaced by annotations, the 4th one affects generated code, 
> still using a (simpified) XML for that.
> 
> 
> Diffs
> -
> 
>   core/src/main/java/org/apache/oozie/command/XCommand.java 7b8f47cc 
>   core/src/main/java/org/apache/oozie/util/Instrumentation.java 8bcb64c1 
>   findbugs-filter.xml 133178f0 
>   fluent-job/fluent-job-api/pom.xml 4c9b8533 
>   
> fluent-job/fluent-job-api/src/main/java/org/apache/oozie/fluentjob/api/GraphVisualization.java
>  437d6b3b 
>   fluent-job/pom.xml 5b24c911 
>   pom.xml 92358aa2 
> 
> 
> Diff: https://reviews.apache.org/r/68255/diff/1/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Andras Salamon
> 
>

Re: Review Request 68255: OOZIE 3314 - Remove findbugs-filter.xml and convert its contents to annotations

[Andras Salamon]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68255/
---

(Updated Aug. 8, 2018, 9:15 a.m.)


Review request for oozie, András Piros and Kinga Marton.


Changes
---

Two of the annotations are eliminated.


Repository: oozie-git


Description
---

3 excptions replaced by annotations, the 4th one affects generated code, still 
using a (simpified) XML for that.


Diffs (updated)
-

  core/src/main/java/org/apache/oozie/command/XCommand.java 7b8f47cc 
  core/src/main/java/org/apache/oozie/command/coord/CoordinatorXCommand.java 
18937a4b 
  core/src/main/java/org/apache/oozie/command/wf/WorkflowXCommand.java 87d7e77f 
  core/src/main/java/org/apache/oozie/util/Instrumentation.java 8bcb64c1 
  findbugs-filter.xml 133178f0 
  fluent-job/fluent-job-api/pom.xml 4c9b8533 
  
fluent-job/fluent-job-api/src/main/java/org/apache/oozie/fluentjob/api/GraphVisualization.java
 437d6b3b 
  fluent-job/pom.xml 5b24c911 
  pom.xml 92358aa2 


Diff: https://reviews.apache.org/r/68255/diff/2/

Changes: https://reviews.apache.org/r/68255/diff/1-2/


Testing
---


Thanks,

Andras Salamon

[jira] [Commented] (OOZIE-3321) PySpark example fails

[Andras Piros (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-3321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16572917#comment-16572917
 ] 

Andras Piros commented on OOZIE-3321:
-

Thanks for the contribution [~daniel.becker]! +1

> PySpark example fails
> -
>
> Key: OOZIE-3321
> URL: https://issues.apache.org/jira/browse/OOZIE-3321
> Project: Oozie
>  Issue Type: Bug
>  Components: examples
>Affects Versions: 5.0.0, 5.1.0
>Reporter: Daniel Becker
>Assignee: Daniel Becker
>Priority: Minor
> Attachments: OOZIE-3321.1.patch, OOZIE-3321.2.patch, 
> OOZIE-3321.3.patch
>
>
> The PySpark example fails with the following exception:
> {noformat}
> ACTION[001-180806145017687-oozie-oozi-W@spark-node] Launcher exception: 
> Missing py4j and/or pyspark zip files. Please add them to the lib folder or 
> to the Spark sharelib.
> org.apache.oozie.action.hadoop.OozieActionConfiguratorException: Missing py4j 
> and/or pyspark zip files. Please add them to the lib folder or to the Spark 
> sharelib.
>   at 
> org.apache.oozie.action.hadoop.SparkMain.getMatchingPyFile(SparkMain.java:151)
>   at 
> org.apache.oozie.action.hadoop.SparkMain.createPySparkLibFolder(SparkMain.java:132)
>   at org.apache.oozie.action.hadoop.SparkMain.run(SparkMain.java:77)
>   at 
> org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:101)
>   at org.apache.oozie.action.hadoop.SparkMain.main(SparkMain.java:60)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:498)
>   at 
> org.apache.oozie.action.hadoop.LauncherAM.runActionMain(LauncherAM.java:410)
>   at 
> org.apache.oozie.action.hadoop.LauncherAM.access$300(LauncherAM.java:55)
>   at org.apache.oozie.action.hadoop.LauncherAM$2.run(LauncherAM.java:223)
>   at java.security.AccessController.doPrivileged(Native Method)
>   at javax.security.auth.Subject.doAs(Subject.java:422)
>   at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1692)
>   at org.apache.oozie.action.hadoop.LauncherAM.run(LauncherAM.java:217)
>   at org.apache.oozie.action.hadoop.LauncherAM$1.run(LauncherAM.java:153)
>   at java.security.AccessController.doPrivileged(Native Method)
>   at javax.security.auth.Subject.doAs(Subject.java:422)
>   at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1692)
>   at org.apache.oozie.action.hadoop.LauncherAM.main(LauncherAM.java:141)
> {noformat}
> This is because py4j-0.9-src.zip and pyspark.zip are not available. They are 
> needed for the example to run, so we should either add them to the sharelib 
> or if they are not important enough to be in the sharelib, we should add them 
> to the example's lib directory. Currently these two files can be found in the 
> codebase under sharelib/spark/src/test/resources. We should also decide what 
> to do with these files there, to avoid duplication of resource files.
> If I add the files manually to either the sharelib or the lib directory of 
> the example, the example runs successfilly.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (OOZIE-3321) PySpark example fails

[Andras Piros (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-3321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andras Piros updated OOZIE-3321:

Affects Version/s: (was: 5.1.0)

> PySpark example fails
> -
>
> Key: OOZIE-3321
> URL: https://issues.apache.org/jira/browse/OOZIE-3321
> Project: Oozie
>  Issue Type: Bug
>  Components: examples
>Affects Versions: 5.0.0
>Reporter: Daniel Becker
>Assignee: Daniel Becker
>Priority: Minor
> Attachments: OOZIE-3321.1.patch, OOZIE-3321.2.patch, 
> OOZIE-3321.3.patch
>
>
> The PySpark example fails with the following exception:
> {noformat}
> ACTION[001-180806145017687-oozie-oozi-W@spark-node] Launcher exception: 
> Missing py4j and/or pyspark zip files. Please add them to the lib folder or 
> to the Spark sharelib.
> org.apache.oozie.action.hadoop.OozieActionConfiguratorException: Missing py4j 
> and/or pyspark zip files. Please add them to the lib folder or to the Spark 
> sharelib.
>   at 
> org.apache.oozie.action.hadoop.SparkMain.getMatchingPyFile(SparkMain.java:151)
>   at 
> org.apache.oozie.action.hadoop.SparkMain.createPySparkLibFolder(SparkMain.java:132)
>   at org.apache.oozie.action.hadoop.SparkMain.run(SparkMain.java:77)
>   at 
> org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:101)
>   at org.apache.oozie.action.hadoop.SparkMain.main(SparkMain.java:60)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:498)
>   at 
> org.apache.oozie.action.hadoop.LauncherAM.runActionMain(LauncherAM.java:410)
>   at 
> org.apache.oozie.action.hadoop.LauncherAM.access$300(LauncherAM.java:55)
>   at org.apache.oozie.action.hadoop.LauncherAM$2.run(LauncherAM.java:223)
>   at java.security.AccessController.doPrivileged(Native Method)
>   at javax.security.auth.Subject.doAs(Subject.java:422)
>   at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1692)
>   at org.apache.oozie.action.hadoop.LauncherAM.run(LauncherAM.java:217)
>   at org.apache.oozie.action.hadoop.LauncherAM$1.run(LauncherAM.java:153)
>   at java.security.AccessController.doPrivileged(Native Method)
>   at javax.security.auth.Subject.doAs(Subject.java:422)
>   at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1692)
>   at org.apache.oozie.action.hadoop.LauncherAM.main(LauncherAM.java:141)
> {noformat}
> This is because py4j-0.9-src.zip and pyspark.zip are not available. They are 
> needed for the example to run, so we should either add them to the sharelib 
> or if they are not important enough to be in the sharelib, we should add them 
> to the example's lib directory. Currently these two files can be found in the 
> codebase under sharelib/spark/src/test/resources. We should also decide what 
> to do with these files there, to avoid duplication of resource files.
> If I add the files manually to either the sharelib or the lib directory of 
> the example, the example runs successfilly.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Review Request 68255: OOZIE 3314 - Remove findbugs-filter.xml and convert its contents to annotations

[András Piros via Review Board]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68255/#review206973
---


Ship it!




Ship It!

- András Piros


On Aug. 8, 2018, 9:15 a.m., Andras Salamon wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/68255/
> ---
> 
> (Updated Aug. 8, 2018, 9:15 a.m.)
> 
> 
> Review request for oozie, András Piros and Kinga Marton.
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> 3 excptions replaced by annotations, the 4th one affects generated code, 
> still using a (simpified) XML for that.
> 
> 
> Diffs
> -
> 
>   core/src/main/java/org/apache/oozie/command/XCommand.java 7b8f47cc 
>   core/src/main/java/org/apache/oozie/command/coord/CoordinatorXCommand.java 
> 18937a4b 
>   core/src/main/java/org/apache/oozie/command/wf/WorkflowXCommand.java 
> 87d7e77f 
>   core/src/main/java/org/apache/oozie/util/Instrumentation.java 8bcb64c1 
>   findbugs-filter.xml 133178f0 
>   fluent-job/fluent-job-api/pom.xml 4c9b8533 
>   
> fluent-job/fluent-job-api/src/main/java/org/apache/oozie/fluentjob/api/GraphVisualization.java
>  437d6b3b 
>   fluent-job/pom.xml 5b24c911 
>   pom.xml 92358aa2 
> 
> 
> Diff: https://reviews.apache.org/r/68255/diff/2/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Andras Salamon
> 
>

[jira] [Commented] (OOZIE-3314) Remove findbugs-filter.xml and convert its contents to annotations

[Andras Piros (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-3314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16572928#comment-16572928
 ] 

Andras Piros commented on OOZIE-3314:
-

Thanks for the contribution [~asalamon74]! +1 (pending Jenkins for path 2)

> Remove findbugs-filter.xml and convert its contents to annotations
> --
>
> Key: OOZIE-3314
> URL: https://issues.apache.org/jira/browse/OOZIE-3314
> Project: Oozie
>  Issue Type: Bug
>  Components: core
>Reporter: Peter Bacsko
>Assignee: Andras Salamon
>Priority: Major
> Attachments: OOZIE-3314-1.patch, OOZIE-3314-2.patch
>
>
> In oozie-core, we have a file called "findbugs-filter.xml" which tells 
> findbugs that it shluld ignore certain problems in a couple of classes.
> However if we try to compile a sub-module or run findbugs directly (let's say 
> in a sharelib project), the build will fail because it won't be able to open 
> findbugs.xml. It's not straightforward how to define the path of the XML in a 
> way that it makes this xml accessible regardless of what module you're 
> compiling.
> It's better to just convert its contents to annotations - especially since we 
> already use this method too.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (OOZIE-3304) Parsing sharelib timestamps is not threadsafe

[Mate Juhasz (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mate Juhasz updated OOZIE-3304:
---
Attachment: OOZIE-3304-004.patch

> Parsing sharelib timestamps is not threadsafe
> -
>
> Key: OOZIE-3304
> URL: https://issues.apache.org/jira/browse/OOZIE-3304
> Project: Oozie
>  Issue Type: Bug
>  Components: core
>Affects Versions: 5.0.0b1, 4.3.1
>Reporter: Denes Bodo
>Assignee: Denes Bodo
>Priority: Critical
>  Labels: usability
> Attachments: OOZIE-3304-001.patch, OOZIE-3304-002.patch, 
> OOZIE-3304-003.patch, OOZIE-3304-004.patch
>
>
> In rare cases the following Exception can be read in log files when an action 
> fails:
> {code:java}
> org.apache.oozie.action.ActionExecutorException: NumberFormatException: 
> multiple points
>   at 
> org.apache.oozie.action.ActionExecutor.convertException(ActionExecutor.java:446)
>   at 
> org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:1271)
>   at 
> org.apache.oozie.action.hadoop.JavaActionExecutor.start(JavaActionExecutor.java:1472)
>   at 
> org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:234)
>   at 
> org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:65)
>   at org.apache.oozie.command.XCommand.call(XCommand.java:287)
>   at 
> org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:332)
>   at 
> org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:261)
>   at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>   at 
> org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:179)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>   at java.lang.Thread.run(Thread.java:748)
> Caused by: java.lang.NumberFormatException: multiple points
>   at 
> sun.misc.FloatingDecimal.readJavaFormatString(FloatingDecimal.java:1890)
>   at sun.misc.FloatingDecimal.parseDouble(FloatingDecimal.java:110)
>   at java.lang.Double.parseDouble(Double.java:538)
>   at java.text.DigitList.getDouble(DigitList.java:169)
>   at java.text.DecimalFormat.parse(DecimalFormat.java:2056)
>   at java.text.SimpleDateFormat.subParse(SimpleDateFormat.java:2160)
>   at java.text.SimpleDateFormat.parse(SimpleDateFormat.java:1514)
>   at java.text.DateFormat.parse(DateFormat.java:364)
>   at 
> org.apache.oozie.service.ShareLibService.getLatestLibPath(ShareLibService.java:727)
>   at 
> org.apache.oozie.service.ShareLibService.getShareLibRootPath(ShareLibService.java:595)
>   at 
> org.apache.oozie.action.hadoop.JavaActionExecutor.getSharelibRoot(JavaActionExecutor.java:1297)
>   at 
> org.apache.oozie.action.hadoop.JavaActionExecutor.initShareLibExcluder(JavaActionExecutor.java:858)
>   at 
> org.apache.oozie.action.hadoop.JavaActionExecutor.setLibFilesArchives(JavaActionExecutor.java:866)
>   at 
> org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:1156)
>   ... 11 more
> {code}
> or
> {code:java}
> 2018-07-12 04:48:52,649  WARN ForkedActionStartXCommand:523 - 
> SERVER[ctr-e138-1518143905142-410551-01-03.hwx.site] USER[user] GROUP[-] 
> TOKEN[] APP[demo-wf] JOB[023-180712043119670-oozie-oozi-W] 
> ACTION[023-180712043119670-oozie-oozi-W@streaming-node] Error starting 
> action [streaming-node]. ErrorType [ERROR], ErrorCode 
> [NumberFormatException], Message [NumberFormatException: For input string: ""]
> org.apache.oozie.action.ActionExecutorException: NumberFormatException: For 
> input string: ""
>   at 
> org.apache.oozie.action.ActionExecutor.convertException(ActionExecutor.java:446)
>   at 
> org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:1271)
>   at 
> org.apache.oozie.action.hadoop.JavaActionExecutor.start(JavaActionExecutor.java:1472)
>   at 
> org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:234)
>   at 
> org.apache.oozie.command.wf.ForkedActionStartXCommand.execute(ForkedActionStartXCommand.java:41)
>   at 
> org.apache.oozie.command.wf.ForkedActionStartXCommand.execute(ForkedActionStartXCommand.java:30)
>   at org.apache.oozie.command.XCommand.call(XCommand.java:287)
>   at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>   at 
> org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:179)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPool

[jira] [Updated] (OOZIE-3304) Parsing sharelib timestamps is not threadsafe

[Mate Juhasz (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mate Juhasz updated OOZIE-3304:
---
Attachment: (was: OOZIE-3304-004.patch)

> Parsing sharelib timestamps is not threadsafe
> -
>
> Key: OOZIE-3304
> URL: https://issues.apache.org/jira/browse/OOZIE-3304
> Project: Oozie
>  Issue Type: Bug
>  Components: core
>Affects Versions: 5.0.0b1, 4.3.1
>Reporter: Denes Bodo
>Assignee: Denes Bodo
>Priority: Critical
>  Labels: usability
> Attachments: OOZIE-3304-001.patch, OOZIE-3304-002.patch, 
> OOZIE-3304-003.patch
>
>
> In rare cases the following Exception can be read in log files when an action 
> fails:
> {code:java}
> org.apache.oozie.action.ActionExecutorException: NumberFormatException: 
> multiple points
>   at 
> org.apache.oozie.action.ActionExecutor.convertException(ActionExecutor.java:446)
>   at 
> org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:1271)
>   at 
> org.apache.oozie.action.hadoop.JavaActionExecutor.start(JavaActionExecutor.java:1472)
>   at 
> org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:234)
>   at 
> org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:65)
>   at org.apache.oozie.command.XCommand.call(XCommand.java:287)
>   at 
> org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:332)
>   at 
> org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:261)
>   at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>   at 
> org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:179)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>   at java.lang.Thread.run(Thread.java:748)
> Caused by: java.lang.NumberFormatException: multiple points
>   at 
> sun.misc.FloatingDecimal.readJavaFormatString(FloatingDecimal.java:1890)
>   at sun.misc.FloatingDecimal.parseDouble(FloatingDecimal.java:110)
>   at java.lang.Double.parseDouble(Double.java:538)
>   at java.text.DigitList.getDouble(DigitList.java:169)
>   at java.text.DecimalFormat.parse(DecimalFormat.java:2056)
>   at java.text.SimpleDateFormat.subParse(SimpleDateFormat.java:2160)
>   at java.text.SimpleDateFormat.parse(SimpleDateFormat.java:1514)
>   at java.text.DateFormat.parse(DateFormat.java:364)
>   at 
> org.apache.oozie.service.ShareLibService.getLatestLibPath(ShareLibService.java:727)
>   at 
> org.apache.oozie.service.ShareLibService.getShareLibRootPath(ShareLibService.java:595)
>   at 
> org.apache.oozie.action.hadoop.JavaActionExecutor.getSharelibRoot(JavaActionExecutor.java:1297)
>   at 
> org.apache.oozie.action.hadoop.JavaActionExecutor.initShareLibExcluder(JavaActionExecutor.java:858)
>   at 
> org.apache.oozie.action.hadoop.JavaActionExecutor.setLibFilesArchives(JavaActionExecutor.java:866)
>   at 
> org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:1156)
>   ... 11 more
> {code}
> or
> {code:java}
> 2018-07-12 04:48:52,649  WARN ForkedActionStartXCommand:523 - 
> SERVER[ctr-e138-1518143905142-410551-01-03.hwx.site] USER[user] GROUP[-] 
> TOKEN[] APP[demo-wf] JOB[023-180712043119670-oozie-oozi-W] 
> ACTION[023-180712043119670-oozie-oozi-W@streaming-node] Error starting 
> action [streaming-node]. ErrorType [ERROR], ErrorCode 
> [NumberFormatException], Message [NumberFormatException: For input string: ""]
> org.apache.oozie.action.ActionExecutorException: NumberFormatException: For 
> input string: ""
>   at 
> org.apache.oozie.action.ActionExecutor.convertException(ActionExecutor.java:446)
>   at 
> org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:1271)
>   at 
> org.apache.oozie.action.hadoop.JavaActionExecutor.start(JavaActionExecutor.java:1472)
>   at 
> org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:234)
>   at 
> org.apache.oozie.command.wf.ForkedActionStartXCommand.execute(ForkedActionStartXCommand.java:41)
>   at 
> org.apache.oozie.command.wf.ForkedActionStartXCommand.execute(ForkedActionStartXCommand.java:30)
>   at org.apache.oozie.command.XCommand.call(XCommand.java:287)
>   at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>   at 
> org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:179)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.ja

[jira] [Updated] (OOZIE-3304) Parsing sharelib timestamps is not threadsafe

[Mate Juhasz (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mate Juhasz updated OOZIE-3304:
---
Attachment: OOZIE-3304-004.patch

> Parsing sharelib timestamps is not threadsafe
> -
>
> Key: OOZIE-3304
> URL: https://issues.apache.org/jira/browse/OOZIE-3304
> Project: Oozie
>  Issue Type: Bug
>  Components: core
>Affects Versions: 5.0.0b1, 4.3.1
>Reporter: Denes Bodo
>Assignee: Denes Bodo
>Priority: Critical
>  Labels: usability
> Attachments: OOZIE-3304-001.patch, OOZIE-3304-002.patch, 
> OOZIE-3304-003.patch, OOZIE-3304-004.patch
>
>
> In rare cases the following Exception can be read in log files when an action 
> fails:
> {code:java}
> org.apache.oozie.action.ActionExecutorException: NumberFormatException: 
> multiple points
>   at 
> org.apache.oozie.action.ActionExecutor.convertException(ActionExecutor.java:446)
>   at 
> org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:1271)
>   at 
> org.apache.oozie.action.hadoop.JavaActionExecutor.start(JavaActionExecutor.java:1472)
>   at 
> org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:234)
>   at 
> org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:65)
>   at org.apache.oozie.command.XCommand.call(XCommand.java:287)
>   at 
> org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:332)
>   at 
> org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:261)
>   at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>   at 
> org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:179)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>   at java.lang.Thread.run(Thread.java:748)
> Caused by: java.lang.NumberFormatException: multiple points
>   at 
> sun.misc.FloatingDecimal.readJavaFormatString(FloatingDecimal.java:1890)
>   at sun.misc.FloatingDecimal.parseDouble(FloatingDecimal.java:110)
>   at java.lang.Double.parseDouble(Double.java:538)
>   at java.text.DigitList.getDouble(DigitList.java:169)
>   at java.text.DecimalFormat.parse(DecimalFormat.java:2056)
>   at java.text.SimpleDateFormat.subParse(SimpleDateFormat.java:2160)
>   at java.text.SimpleDateFormat.parse(SimpleDateFormat.java:1514)
>   at java.text.DateFormat.parse(DateFormat.java:364)
>   at 
> org.apache.oozie.service.ShareLibService.getLatestLibPath(ShareLibService.java:727)
>   at 
> org.apache.oozie.service.ShareLibService.getShareLibRootPath(ShareLibService.java:595)
>   at 
> org.apache.oozie.action.hadoop.JavaActionExecutor.getSharelibRoot(JavaActionExecutor.java:1297)
>   at 
> org.apache.oozie.action.hadoop.JavaActionExecutor.initShareLibExcluder(JavaActionExecutor.java:858)
>   at 
> org.apache.oozie.action.hadoop.JavaActionExecutor.setLibFilesArchives(JavaActionExecutor.java:866)
>   at 
> org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:1156)
>   ... 11 more
> {code}
> or
> {code:java}
> 2018-07-12 04:48:52,649  WARN ForkedActionStartXCommand:523 - 
> SERVER[ctr-e138-1518143905142-410551-01-03.hwx.site] USER[user] GROUP[-] 
> TOKEN[] APP[demo-wf] JOB[023-180712043119670-oozie-oozi-W] 
> ACTION[023-180712043119670-oozie-oozi-W@streaming-node] Error starting 
> action [streaming-node]. ErrorType [ERROR], ErrorCode 
> [NumberFormatException], Message [NumberFormatException: For input string: ""]
> org.apache.oozie.action.ActionExecutorException: NumberFormatException: For 
> input string: ""
>   at 
> org.apache.oozie.action.ActionExecutor.convertException(ActionExecutor.java:446)
>   at 
> org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:1271)
>   at 
> org.apache.oozie.action.hadoop.JavaActionExecutor.start(JavaActionExecutor.java:1472)
>   at 
> org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:234)
>   at 
> org.apache.oozie.command.wf.ForkedActionStartXCommand.execute(ForkedActionStartXCommand.java:41)
>   at 
> org.apache.oozie.command.wf.ForkedActionStartXCommand.execute(ForkedActionStartXCommand.java:30)
>   at org.apache.oozie.command.XCommand.call(XCommand.java:287)
>   at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>   at 
> org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:179)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPool

[jira] [Commented] (OOZIE-3304) Parsing sharelib timestamps is not threadsafe

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16572944#comment-16572944
 ] 

Hadoop QA commented on OOZIE-3304:
--

PreCommit-OOZIE-Build started


> Parsing sharelib timestamps is not threadsafe
> -
>
> Key: OOZIE-3304
> URL: https://issues.apache.org/jira/browse/OOZIE-3304
> Project: Oozie
>  Issue Type: Bug
>  Components: core
>Affects Versions: 5.0.0b1, 4.3.1
>Reporter: Denes Bodo
>Assignee: Denes Bodo
>Priority: Critical
>  Labels: usability
> Attachments: OOZIE-3304-001.patch, OOZIE-3304-002.patch, 
> OOZIE-3304-003.patch, OOZIE-3304-004.patch
>
>
> In rare cases the following Exception can be read in log files when an action 
> fails:
> {code:java}
> org.apache.oozie.action.ActionExecutorException: NumberFormatException: 
> multiple points
>   at 
> org.apache.oozie.action.ActionExecutor.convertException(ActionExecutor.java:446)
>   at 
> org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:1271)
>   at 
> org.apache.oozie.action.hadoop.JavaActionExecutor.start(JavaActionExecutor.java:1472)
>   at 
> org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:234)
>   at 
> org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:65)
>   at org.apache.oozie.command.XCommand.call(XCommand.java:287)
>   at 
> org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:332)
>   at 
> org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:261)
>   at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>   at 
> org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:179)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>   at java.lang.Thread.run(Thread.java:748)
> Caused by: java.lang.NumberFormatException: multiple points
>   at 
> sun.misc.FloatingDecimal.readJavaFormatString(FloatingDecimal.java:1890)
>   at sun.misc.FloatingDecimal.parseDouble(FloatingDecimal.java:110)
>   at java.lang.Double.parseDouble(Double.java:538)
>   at java.text.DigitList.getDouble(DigitList.java:169)
>   at java.text.DecimalFormat.parse(DecimalFormat.java:2056)
>   at java.text.SimpleDateFormat.subParse(SimpleDateFormat.java:2160)
>   at java.text.SimpleDateFormat.parse(SimpleDateFormat.java:1514)
>   at java.text.DateFormat.parse(DateFormat.java:364)
>   at 
> org.apache.oozie.service.ShareLibService.getLatestLibPath(ShareLibService.java:727)
>   at 
> org.apache.oozie.service.ShareLibService.getShareLibRootPath(ShareLibService.java:595)
>   at 
> org.apache.oozie.action.hadoop.JavaActionExecutor.getSharelibRoot(JavaActionExecutor.java:1297)
>   at 
> org.apache.oozie.action.hadoop.JavaActionExecutor.initShareLibExcluder(JavaActionExecutor.java:858)
>   at 
> org.apache.oozie.action.hadoop.JavaActionExecutor.setLibFilesArchives(JavaActionExecutor.java:866)
>   at 
> org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:1156)
>   ... 11 more
> {code}
> or
> {code:java}
> 2018-07-12 04:48:52,649  WARN ForkedActionStartXCommand:523 - 
> SERVER[ctr-e138-1518143905142-410551-01-03.hwx.site] USER[user] GROUP[-] 
> TOKEN[] APP[demo-wf] JOB[023-180712043119670-oozie-oozi-W] 
> ACTION[023-180712043119670-oozie-oozi-W@streaming-node] Error starting 
> action [streaming-node]. ErrorType [ERROR], ErrorCode 
> [NumberFormatException], Message [NumberFormatException: For input string: ""]
> org.apache.oozie.action.ActionExecutorException: NumberFormatException: For 
> input string: ""
>   at 
> org.apache.oozie.action.ActionExecutor.convertException(ActionExecutor.java:446)
>   at 
> org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:1271)
>   at 
> org.apache.oozie.action.hadoop.JavaActionExecutor.start(JavaActionExecutor.java:1472)
>   at 
> org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:234)
>   at 
> org.apache.oozie.command.wf.ForkedActionStartXCommand.execute(ForkedActionStartXCommand.java:41)
>   at 
> org.apache.oozie.command.wf.ForkedActionStartXCommand.execute(ForkedActionStartXCommand.java:30)
>   at org.apache.oozie.command.XCommand.call(XCommand.java:287)
>   at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>   at 
> org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:179)
>   at 
> java.util.co

Re: Review Request 59620: This review board request is for an action to provide a Git action for Oozie

[Peter Bacsko via Review Board]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59620/#review206974
---




core/src/main/java/org/apache/oozie/action/hadoop/GitActionExecutor.java
Lines 152 (patched)


Better naming suggested: ActionConfVerifier



core/src/main/java/org/apache/oozie/action/hadoop/GitActionExecutor.java
Lines 169 (patched)


Check visibility of this class & methods (package private OK?)



sharelib/git/src/main/java/org/apache/oozie/action/hadoop/GitMain.java
Lines 38 (patched)


General thoughts: this class has quite a few protected methods. We have to 
think about whether it's necessary or not. If the class is not going to be 
subclassed, making them "private" is preferable.



sharelib/git/src/main/java/org/apache/oozie/action/hadoop/GitMain.java
Lines 43 (patched)


I'd prefer this as being private, with having a package private setter 
method.



sharelib/git/src/main/java/org/apache/oozie/action/hadoop/GitOperations.java
Lines 125 (patched)


This string can be placed directly in the constructor.



sharelib/git/src/test/java/org/apache/oozie/action/hadoop/GitServer.java
Lines 74 (patched)


At least a warning/error message would be good here.


- Peter Bacsko


On aug. 3, 2018, 10 du, Clay B. wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/59620/
> ---
> 
> (Updated aug. 3, 2018, 10 du)
> 
> 
> Review request for oozie and András Piros.
> 
> 
> Bugs: OOZIE-2877
> https://issues.apache.org/jira/browse/OOZIE-2877
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> OOZIE-2877 - Oozie Git Action
> 
> 
> Diffs
> -
> 
>   client/src/main/resources/git-action-1.0.xsd PRE-CREATION 
>   core/src/main/java/org/apache/oozie/action/hadoop/GitActionExecutor.java 
> PRE-CREATION 
>   core/src/main/resources/oozie-default.xml b69d2c9 
>   core/src/test/java/org/apache/oozie/test/XTestCase.java 661970d 
>   docs/src/site/twiki/WorkflowFunctionalSpec.twiki 76cbe21 
>   examples/src/main/apps/git/job.properties PRE-CREATION 
>   examples/src/main/apps/git/workflow.xml PRE-CREATION 
>   fluent-job/fluent-job-api/pom.xml 4c9b853 
>   
> fluent-job/fluent-job-api/src/main/java/org/apache/oozie/fluentjob/api/action/GitAction.java
>  PRE-CREATION 
>   
> fluent-job/fluent-job-api/src/main/java/org/apache/oozie/fluentjob/api/action/GitActionBuilder.java
>  PRE-CREATION 
>   
> fluent-job/fluent-job-api/src/main/java/org/apache/oozie/fluentjob/api/mapping/ExplicitNodeConverter.java
>  7bb82e5 
>   
> fluent-job/fluent-job-api/src/main/java/org/apache/oozie/fluentjob/api/mapping/GitConfigurationConverter.java
>  PRE-CREATION 
>   
> fluent-job/fluent-job-api/src/main/java/org/apache/oozie/fluentjob/api/mapping/GitPrepareConverter.java
>  PRE-CREATION 
>   
> fluent-job/fluent-job-api/src/main/java/org/apache/oozie/fluentjob/api/serialization/WorkflowMarshaller.java
>  ec56554 
>   fluent-job/fluent-job-api/src/main/resources/action_mappings.xml a5f890e 
>   fluent-job/fluent-job-api/src/main/xjb/bindings.xml 48f6890 
>   pom.xml 92358aa 
>   sharelib/git/pom.xml PRE-CREATION 
>   sharelib/git/src/main/java/org/apache/oozie/action/hadoop/GitMain.java 
> PRE-CREATION 
>   
> sharelib/git/src/main/java/org/apache/oozie/action/hadoop/GitOperations.java 
> PRE-CREATION 
>   sharelib/git/src/test/java/org/apache/oozie/action/hadoop/GitServer.java 
> PRE-CREATION 
>   
> sharelib/git/src/test/java/org/apache/oozie/action/hadoop/TestGitActionExecutor.java
>  PRE-CREATION 
>   sharelib/git/src/test/java/org/apache/oozie/action/hadoop/TestGitMain.java 
> PRE-CREATION 
>   
> sharelib/git/src/test/java/org/apache/oozie/action/hadoop/TestIntegrationGitActionExecutor.java
>  PRE-CREATION 
>   sharelib/pom.xml 6a0864d 
>   src/main/assemblies/sharelib.xml 07dc69c 
>   webapp/pom.xml fd3f89f 
> 
> 
> Diff: https://reviews.apache.org/r/59620/diff/10/
> 
> 
> Testing
> ---
> 
> Tested using unit and integration tests. Still need to:
> * Test on a cluster
> * Test with an authenticated SSH hosted Git repo
> 
> Sumitted a request to the JGit community as their branch pulling code seems 
> to have an 
> [issue](https://dev.eclipse.org/mhonarc/lists/jgit-dev/msg03343.html).
> 
> Andras has tested against his patch 011 with results at 
> https://issues.apache.org/jira/browse/OOZIE-2877?focusedCommentId=16459523&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment

Re: Review Request 59620: This review board request is for an action to provide a Git action for Oozie

[Peter Bacsko via Review Board]

> On júl. 2, 2018, 12:46 du, Peter Bacsko wrote:
> > sharelib/git/src/main/java/org/apache/oozie/action/hadoop/GitMain.java
> > Lines 41 (patched)
> > 
> >
> > We don't use XLog inside LauncherMain, only Sysout. This will change in 
> > the future, but for now, let's stick to the conventions that we have 
> > already.
> 
> Clay B. wrote:
> I have left the help methods to leave logging level intention in place 
> but removed all references to XLog. Would you prefer the helpers to go too?

We haven't settled down on a logging solution. Geza suggested JDK loggers (so 
that they don't conflict with log4j/log4j2/logback, whatever we migrate to), 
but it's not finalized yet.


- Peter


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59620/#review205636
---


On aug. 3, 2018, 10 du, Clay B. wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/59620/
> ---
> 
> (Updated aug. 3, 2018, 10 du)
> 
> 
> Review request for oozie and András Piros.
> 
> 
> Bugs: OOZIE-2877
> https://issues.apache.org/jira/browse/OOZIE-2877
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> OOZIE-2877 - Oozie Git Action
> 
> 
> Diffs
> -
> 
>   client/src/main/resources/git-action-1.0.xsd PRE-CREATION 
>   core/src/main/java/org/apache/oozie/action/hadoop/GitActionExecutor.java 
> PRE-CREATION 
>   core/src/main/resources/oozie-default.xml b69d2c9 
>   core/src/test/java/org/apache/oozie/test/XTestCase.java 661970d 
>   docs/src/site/twiki/WorkflowFunctionalSpec.twiki 76cbe21 
>   examples/src/main/apps/git/job.properties PRE-CREATION 
>   examples/src/main/apps/git/workflow.xml PRE-CREATION 
>   fluent-job/fluent-job-api/pom.xml 4c9b853 
>   
> fluent-job/fluent-job-api/src/main/java/org/apache/oozie/fluentjob/api/action/GitAction.java
>  PRE-CREATION 
>   
> fluent-job/fluent-job-api/src/main/java/org/apache/oozie/fluentjob/api/action/GitActionBuilder.java
>  PRE-CREATION 
>   
> fluent-job/fluent-job-api/src/main/java/org/apache/oozie/fluentjob/api/mapping/ExplicitNodeConverter.java
>  7bb82e5 
>   
> fluent-job/fluent-job-api/src/main/java/org/apache/oozie/fluentjob/api/mapping/GitConfigurationConverter.java
>  PRE-CREATION 
>   
> fluent-job/fluent-job-api/src/main/java/org/apache/oozie/fluentjob/api/mapping/GitPrepareConverter.java
>  PRE-CREATION 
>   
> fluent-job/fluent-job-api/src/main/java/org/apache/oozie/fluentjob/api/serialization/WorkflowMarshaller.java
>  ec56554 
>   fluent-job/fluent-job-api/src/main/resources/action_mappings.xml a5f890e 
>   fluent-job/fluent-job-api/src/main/xjb/bindings.xml 48f6890 
>   pom.xml 92358aa 
>   sharelib/git/pom.xml PRE-CREATION 
>   sharelib/git/src/main/java/org/apache/oozie/action/hadoop/GitMain.java 
> PRE-CREATION 
>   
> sharelib/git/src/main/java/org/apache/oozie/action/hadoop/GitOperations.java 
> PRE-CREATION 
>   sharelib/git/src/test/java/org/apache/oozie/action/hadoop/GitServer.java 
> PRE-CREATION 
>   
> sharelib/git/src/test/java/org/apache/oozie/action/hadoop/TestGitActionExecutor.java
>  PRE-CREATION 
>   sharelib/git/src/test/java/org/apache/oozie/action/hadoop/TestGitMain.java 
> PRE-CREATION 
>   
> sharelib/git/src/test/java/org/apache/oozie/action/hadoop/TestIntegrationGitActionExecutor.java
>  PRE-CREATION 
>   sharelib/pom.xml 6a0864d 
>   src/main/assemblies/sharelib.xml 07dc69c 
>   webapp/pom.xml fd3f89f 
> 
> 
> Diff: https://reviews.apache.org/r/59620/diff/10/
> 
> 
> Testing
> ---
> 
> Tested using unit and integration tests. Still need to:
> * Test on a cluster
> * Test with an authenticated SSH hosted Git repo
> 
> Sumitted a request to the JGit community as their branch pulling code seems 
> to have an 
> [issue](https://dev.eclipse.org/mhonarc/lists/jgit-dev/msg03343.html).
> 
> Andras has tested against his patch 011 with results at 
> https://issues.apache.org/jira/browse/OOZIE-2877?focusedCommentId=16459523&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16459523
> 
> 
> File Attachments
> 
> 
> 0001-OOZIE-2877-Oozie-Git-Action.patch
>   
> https://reviews.apache.org/media/uploaded/files/2017/05/29/24f90a78-3dc1-49fe-bf29-5927a3cd5e72__0001-OOZIE-2877-Oozie-Git-Action.patch
> Patch
>   
> https://reviews.apache.org/media/uploaded/files/2017/05/29/dd23dd72-67e0-456f-9b52-e566d8d17d16__0001-OOZIE-2877-Oozie-Git-Action.patch
> 
> 
> Thanks,
> 
> Clay B.
> 
>

Failed: OOZIE-3314 PreCommit Build #737

[Apache Jenkins Server]

Jira: https://issues.apache.org/jira/browse/OOZIE-3314
Build: https://builds.apache.org/job/PreCommit-OOZIE-Build/737/

###
## LAST 100 LINES OF THE CONSOLE 
###
[...truncated 1.75 MB...]
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [sharelib/hive].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [sharelib/spark].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [client].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [examples].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [docs].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [server].
[INFO] There are no new bugs found totally].
[TRACE] FindBugs diffs checked and reports created
[TRACE] Summary file size is 2451 bytes
[TRACE] Full summary file size is 1399 bytes
[TRACE] File 
[/home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/tmp/FINDBUGS_DIFF/diff/findbugs-diff-0.1.0-all.jar]
 removed
[TRACE] File 
[/home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/tmp/FINDBUGS_DIFF/diff/findbugs-diff-0.1.0-all.jar.md5sum]
 removed
  Running test-patch task BACKWARDS_COMPATIBILITY
  Running test-patch task TESTS
  Running test-patch task DISTRO


Testing JIRA OOZIE-3314

Cleaning local git workspace



+1 PATCH_APPLIES
+1 CLEAN
-1 RAW_PATCH_ANALYSIS
+1 the patch does not introduce any @author tags
+1 the patch does not introduce any tabs
+1 the patch does not introduce any trailing spaces
+1 the patch does not introduce any line longer than 132
-1 the patch does not add/modify any testcase
+1 RAT
+1 the patch does not seem to introduce new RAT warnings
+1 JAVADOC
+1 JAVADOC
+1 the patch does not seem to introduce new Javadoc warning(s)
+1 the patch does not seem to introduce new Javadoc error(s)
ERROR: the current HEAD has 2 Javadoc error(s)
+1 COMPILE
+1 HEAD compiles
+1 patch compiles
+1 the patch does not seem to introduce new javac warnings
+1 There are no new bugs found in total.
 +1 There are no new bugs found in [webapp].
 +1 There are no new bugs found in [core].
 +1 There are no new bugs found in [tools].
 +1 There are no new bugs found in [fluent-job/fluent-job-api].
 +1 There are no new bugs found in [sharelib/hive2].
 +1 There are no new bugs found in [sharelib/distcp].
 +1 There are no new bugs found in [sharelib/hcatalog].
 +1 There are no new bugs found in [sharelib/streaming].
 +1 There are no new bugs found in [sharelib/sqoop].
 +1 There are no new bugs found in [sharelib/oozie].
 +1 There are no new bugs found in [sharelib/pig].
 +1 There are no new bugs found in [sharelib/hive].
 +1 There are no new bugs found in [sharelib/spark].
 +1 There are no new bugs found in [client].
 +1 There are no new bugs found in [examples].
 +1 There are no new bugs found in [docs].
 +1 There are no new bugs found in [server].
+1 BACKWARDS_COMPATIBILITY
+1 the patch does not change any JPA Entity/Colum/Basic/Lob/Transient 
annotations
+1 the patch does not modify JPA files
+1 TESTS
Tests run: 2914
Tests failed at first run:
TestCoordActionsKillXCommand#testActionKillCommandActionNumbers
For the complete list of flaky tests, see TEST-SUMMARY-FULL files.
+1 DISTRO
+1 distro tarball builds with the patch 


-1 Overall result, please check the reported -1(s)


The full output of the test-patch run is available at

 https://builds.apache.org/job/PreCommit-OOZIE-Build/737/

Adding comment to JIRA
  % Total% Received % Xferd  Average Speed   TimeTime Time  Current
 Dload  Upload   Total   SpentLeft  Speed
  0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 0  
0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 
0{"self":"https://issues.apache.org/jira/rest/api/2/issue/13176435/comment/16573015","id":"16573015","author":{"self":"https://issues.apache.org/jira/rest/api/2/user?username=hadoopqa","name":"hadoopqa","key":"hadoopqa","avatarUrls":{"48x48":"https://issues.apache.org/jira/secure/useravatar?ownerId=hadoopqa&avatarId=10393","24x24":"https://issues.apache.org/jira/secure/useravatar?size=small&ownerId=hadoopqa&avatarId=10393","16x16":"https://issues.apache.org/jira/secure/useravatar?size=xsmall&ownerId=hadoopqa&avatarId=10393","32x32":"https://issues.apache.org/jira/secure/useravatar?size=medium&ownerId=hadoopqa&avatarId=10393"},"displayName":"Hadoop
 QA"

[jira] [Commented] (OOZIE-3314) Remove findbugs-filter.xml and convert its contents to annotations

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-3314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16573015#comment-16573015
 ] 

Hadoop QA commented on OOZIE-3314:
--


Testing JIRA OOZIE-3314

Cleaning local git workspace



{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:red}-1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than 
132
.{color:red}-1{color} the patch does not add/modify any testcase
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT 
warnings
{color:green}+1 JAVADOC{color}
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc 
warning(s)
.{color:green}+1{color} the patch does not seem to introduce new Javadoc 
error(s)
.{color:red}ERROR{color}: the current HEAD has 2 Javadoc error(s)
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac 
warnings
{color:green}+1{color} There are no new bugs found in total.
. {color:green}+1{color} There are no new bugs found in [webapp].
. {color:green}+1{color} There are no new bugs found in [core].
. {color:green}+1{color} There are no new bugs found in [tools].
. {color:green}+1{color} There are no new bugs found in 
[fluent-job/fluent-job-api].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive2].
. {color:green}+1{color} There are no new bugs found in [sharelib/distcp].
. {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
. {color:green}+1{color} There are no new bugs found in [sharelib/streaming].
. {color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
. {color:green}+1{color} There are no new bugs found in [sharelib/oozie].
. {color:green}+1{color} There are no new bugs found in [sharelib/pig].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive].
. {color:green}+1{color} There are no new bugs found in [sharelib/spark].
. {color:green}+1{color} There are no new bugs found in [client].
. {color:green}+1{color} There are no new bugs found in [examples].
. {color:green}+1{color} There are no new bugs found in [docs].
. {color:green}+1{color} There are no new bugs found in [server].
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA 
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:green}+1 TESTS{color}
.Tests run: 2914
.{color:orange}Tests failed at first run:{color}
TestCoordActionsKillXCommand#testActionKillCommandActionNumbers
.For the complete list of flaky tests, see TEST-SUMMARY-FULL files.
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch 


{color:red}*-1 Overall result, please check the reported -1(s)*{color}


The full output of the test-patch run is available at

. https://builds.apache.org/job/PreCommit-OOZIE-Build/737/



> Remove findbugs-filter.xml and convert its contents to annotations
> --
>
> Key: OOZIE-3314
> URL: https://issues.apache.org/jira/browse/OOZIE-3314
> Project: Oozie
>  Issue Type: Bug
>  Components: core
>Reporter: Peter Bacsko
>Assignee: Andras Salamon
>Priority: Major
> Attachments: OOZIE-3314-1.patch, OOZIE-3314-2.patch
>
>
> In oozie-core, we have a file called "findbugs-filter.xml" which tells 
> findbugs that it shluld ignore certain problems in a couple of classes.
> However if we try to compile a sub-module or run findbugs directly (let's say 
> in a sharelib project), the build will fail because it won't be able to open 
> findbugs.xml. It's not straightforward how to define the path of the XML in a 
> way that it makes this xml accessible regardless of what module you're 
> compiling.
> It's better to just convert its contents to annotations - especially since we 
> already use this method too.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Success: OOZIE-3304 PreCommit Build #738

[Apache Jenkins Server]

Jira: https://issues.apache.org/jira/browse/OOZIE-3304
Build: https://builds.apache.org/job/PreCommit-OOZIE-Build/738/

###
## LAST 100 LINES OF THE CONSOLE 
###
[...truncated 1.75 MB...]
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [sharelib/hive].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [sharelib/hcatalog].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [sharelib/sqoop].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [sharelib/oozie].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [sharelib/distcp].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [sharelib/spark].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [client].
[WARN] There are [2] new bugs found in total that would be nice to have fixed.
[TRACE] FindBugs diffs checked and reports created
[TRACE] Summary file size is 2586 bytes
[TRACE] Full summary file size is 1529 bytes
[TRACE] File 
[/home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/tmp/FINDBUGS_DIFF/diff/findbugs-diff-0.1.0-all.jar]
 removed
[TRACE] File 
[/home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/tmp/FINDBUGS_DIFF/diff/findbugs-diff-0.1.0-all.jar.md5sum]
 removed
  Running test-patch task BACKWARDS_COMPATIBILITY
  Running test-patch task TESTS
  Running test-patch task DISTRO


Testing JIRA OOZIE-3304

Cleaning local git workspace



+1 PATCH_APPLIES
+1 CLEAN
+1 RAW_PATCH_ANALYSIS
+1 the patch does not introduce any @author tags
+1 the patch does not introduce any tabs
+1 the patch does not introduce any trailing spaces
+1 the patch does not introduce any line longer than 132
+1 the patch adds/modifies 2 testcase(s)
+1 RAT
+1 the patch does not seem to introduce new RAT warnings
+1 JAVADOC
+1 JAVADOC
+1 the patch does not seem to introduce new Javadoc warning(s)
WARNING: the current HEAD has 100 Javadoc warning(s)
+1 the patch does not seem to introduce new Javadoc error(s)
+1 COMPILE
+1 HEAD compiles
+1 patch compiles
+1 the patch does not seem to introduce new javac warnings
0 There are [2] new bugs found in total that would be nice to have fixed.
 +1 There are no new bugs found in [examples].
 +1 There are no new bugs found in [webapp].
 0 There are [2] new bugs found in [core] that would be nice to have fixed.
 You can find the FindBugs diff here: core/findbugs-new.html
 +1 There are no new bugs found in [tools].
 +1 There are no new bugs found in [fluent-job/fluent-job-api].
 +1 There are no new bugs found in [server].
 +1 There are no new bugs found in [docs].
 +1 There are no new bugs found in [sharelib/hive2].
 +1 There are no new bugs found in [sharelib/pig].
 +1 There are no new bugs found in [sharelib/streaming].
 +1 There are no new bugs found in [sharelib/hive].
 +1 There are no new bugs found in [sharelib/hcatalog].
 +1 There are no new bugs found in [sharelib/sqoop].
 +1 There are no new bugs found in [sharelib/oozie].
 +1 There are no new bugs found in [sharelib/distcp].
 +1 There are no new bugs found in [sharelib/spark].
 +1 There are no new bugs found in [client].
+1 BACKWARDS_COMPATIBILITY
+1 the patch does not change any JPA Entity/Colum/Basic/Lob/Transient 
annotations
+1 the patch does not modify JPA files
+1 TESTS
Tests run: 2915
+1 DISTRO
+1 distro tarball builds with the patch 


+1 Overall result, good!, no -1s

 There is at least one warning, please check

The full output of the test-patch run is available at

 https://builds.apache.org/job/PreCommit-OOZIE-Build/738/

Adding comment to JIRA
  % Total% Received % Xferd  Average Speed   TimeTime Time  Current
 Dload  Upload   Total   SpentLeft  Speed
  0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 
0100  32900 0  100  3290  0   3796 --:--:-- --:--:-- --:--:--  
3794{"self":"https://issues.apache.org/jira/rest/api/2/issue/13172628/comment/16573028","id":"16573028","author":{"self":"https://issues.apache.org/jira/rest/api/2/user?username=hadoopqa","name":"hadoopqa","key":"hadoopqa","avatarUrls":{"48x48":"https://issues.apache.org/jira/secure/useravatar?ownerId=hadoopqa&avatarId=10393","24x24":"https://issues.apache.org/jira/secure/useravatar?size=small&ownerId=hadoopqa&avatarId=10393","16x16":"https://issues.apache.org/jira/secu

[jira] [Commented] (OOZIE-3304) Parsing sharelib timestamps is not threadsafe

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16573028#comment-16573028
 ] 

Hadoop QA commented on OOZIE-3304:
--


Testing JIRA OOZIE-3304

Cleaning local git workspace



{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:green}+1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than 
132
.{color:green}+1{color} the patch adds/modifies 2 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT 
warnings
{color:green}+1 JAVADOC{color}
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc 
warning(s)
.{color:red}WARNING{color}: the current HEAD has 100 Javadoc warning(s)
.{color:green}+1{color} the patch does not seem to introduce new Javadoc 
error(s)
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac 
warnings
{color:orange}0{color} There are [2] new bugs found in total that would be nice 
to have fixed.
. {color:green}+1{color} There are no new bugs found in [examples].
. {color:green}+1{color} There are no new bugs found in [webapp].
. {color:orange}0{color} There are [2] new bugs found in [core] that would be 
nice to have fixed.
. You can find the FindBugs diff here: core/findbugs-new.html
. {color:green}+1{color} There are no new bugs found in [tools].
. {color:green}+1{color} There are no new bugs found in 
[fluent-job/fluent-job-api].
. {color:green}+1{color} There are no new bugs found in [server].
. {color:green}+1{color} There are no new bugs found in [docs].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive2].
. {color:green}+1{color} There are no new bugs found in [sharelib/pig].
. {color:green}+1{color} There are no new bugs found in [sharelib/streaming].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive].
. {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
. {color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
. {color:green}+1{color} There are no new bugs found in [sharelib/oozie].
. {color:green}+1{color} There are no new bugs found in [sharelib/distcp].
. {color:green}+1{color} There are no new bugs found in [sharelib/spark].
. {color:green}+1{color} There are no new bugs found in [client].
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA 
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:green}+1 TESTS{color}
.Tests run: 2915
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch 


{color:green}*+1 Overall result, good!, no -1s*{color}

{color:red}. There is at least one warning, please check{color}

The full output of the test-patch run is available at

. https://builds.apache.org/job/PreCommit-OOZIE-Build/738/



> Parsing sharelib timestamps is not threadsafe
> -
>
> Key: OOZIE-3304
> URL: https://issues.apache.org/jira/browse/OOZIE-3304
> Project: Oozie
>  Issue Type: Bug
>  Components: core
>Affects Versions: 5.0.0b1, 4.3.1
>Reporter: Denes Bodo
>Assignee: Denes Bodo
>Priority: Critical
>  Labels: usability
> Attachments: OOZIE-3304-001.patch, OOZIE-3304-002.patch, 
> OOZIE-3304-003.patch, OOZIE-3304-004.patch
>
>
> In rare cases the following Exception can be read in log files when an action 
> fails:
> {code:java}
> org.apache.oozie.action.ActionExecutorException: NumberFormatException: 
> multiple points
>   at 
> org.apache.oozie.action.ActionExecutor.convertException(ActionExecutor.java:446)
>   at 
> org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:1271)
>   at 
> org.apache.oozie.action.hadoop.JavaActionExecutor.start(JavaActionExecutor.java:1472)
>   at 
> org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:234)
>   at 
> org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:65)
>   at org.apache.oozie.command.XCommand.call(XCommand.java:287)
>   at 
> org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:332)
>   at 
> org.apache.oozie.service.CallableQueueService$Compos

[jira] [Commented] (OOZIE-3304) Parsing sharelib timestamps is not threadsafe

[Mate Juhasz (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16573099#comment-16573099
 ] 

Mate Juhasz commented on OOZIE-3304:


Seems like the SchedulerService caused some trouble with scheduling a sharelib 
purge on the many mocked libs in a separate thread. When its eliminated the 
test execution looks good. Could you take a look at the latest patch? I may 
have added some unwanted static imports as well... :)

> Parsing sharelib timestamps is not threadsafe
> -
>
> Key: OOZIE-3304
> URL: https://issues.apache.org/jira/browse/OOZIE-3304
> Project: Oozie
>  Issue Type: Bug
>  Components: core
>Affects Versions: 5.0.0b1, 4.3.1
>Reporter: Denes Bodo
>Assignee: Denes Bodo
>Priority: Critical
>  Labels: usability
> Attachments: OOZIE-3304-001.patch, OOZIE-3304-002.patch, 
> OOZIE-3304-003.patch, OOZIE-3304-004.patch
>
>
> In rare cases the following Exception can be read in log files when an action 
> fails:
> {code:java}
> org.apache.oozie.action.ActionExecutorException: NumberFormatException: 
> multiple points
>   at 
> org.apache.oozie.action.ActionExecutor.convertException(ActionExecutor.java:446)
>   at 
> org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:1271)
>   at 
> org.apache.oozie.action.hadoop.JavaActionExecutor.start(JavaActionExecutor.java:1472)
>   at 
> org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:234)
>   at 
> org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:65)
>   at org.apache.oozie.command.XCommand.call(XCommand.java:287)
>   at 
> org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:332)
>   at 
> org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:261)
>   at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>   at 
> org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:179)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>   at java.lang.Thread.run(Thread.java:748)
> Caused by: java.lang.NumberFormatException: multiple points
>   at 
> sun.misc.FloatingDecimal.readJavaFormatString(FloatingDecimal.java:1890)
>   at sun.misc.FloatingDecimal.parseDouble(FloatingDecimal.java:110)
>   at java.lang.Double.parseDouble(Double.java:538)
>   at java.text.DigitList.getDouble(DigitList.java:169)
>   at java.text.DecimalFormat.parse(DecimalFormat.java:2056)
>   at java.text.SimpleDateFormat.subParse(SimpleDateFormat.java:2160)
>   at java.text.SimpleDateFormat.parse(SimpleDateFormat.java:1514)
>   at java.text.DateFormat.parse(DateFormat.java:364)
>   at 
> org.apache.oozie.service.ShareLibService.getLatestLibPath(ShareLibService.java:727)
>   at 
> org.apache.oozie.service.ShareLibService.getShareLibRootPath(ShareLibService.java:595)
>   at 
> org.apache.oozie.action.hadoop.JavaActionExecutor.getSharelibRoot(JavaActionExecutor.java:1297)
>   at 
> org.apache.oozie.action.hadoop.JavaActionExecutor.initShareLibExcluder(JavaActionExecutor.java:858)
>   at 
> org.apache.oozie.action.hadoop.JavaActionExecutor.setLibFilesArchives(JavaActionExecutor.java:866)
>   at 
> org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:1156)
>   ... 11 more
> {code}
> or
> {code:java}
> 2018-07-12 04:48:52,649  WARN ForkedActionStartXCommand:523 - 
> SERVER[ctr-e138-1518143905142-410551-01-03.hwx.site] USER[user] GROUP[-] 
> TOKEN[] APP[demo-wf] JOB[023-180712043119670-oozie-oozi-W] 
> ACTION[023-180712043119670-oozie-oozi-W@streaming-node] Error starting 
> action [streaming-node]. ErrorType [ERROR], ErrorCode 
> [NumberFormatException], Message [NumberFormatException: For input string: ""]
> org.apache.oozie.action.ActionExecutorException: NumberFormatException: For 
> input string: ""
>   at 
> org.apache.oozie.action.ActionExecutor.convertException(ActionExecutor.java:446)
>   at 
> org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:1271)
>   at 
> org.apache.oozie.action.hadoop.JavaActionExecutor.start(JavaActionExecutor.java:1472)
>   at 
> org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:234)
>   at 
> org.apache.oozie.command.wf.ForkedActionStartXCommand.execute(ForkedActionStartXCommand.java:41)
>   at 
> org.apache.oozie.command.wf.ForkedActionStartXCommand.execute(ForkedActionStartXCommand.java:30)
>   at or

[jira] [Commented] (OOZIE-3304) Parsing sharelib timestamps is not threadsafe

[Andras Piros (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16573151#comment-16573151
 ] 

Andras Piros commented on OOZIE-3304:
-

Thanks for the contribution [~matijhs]! Nice improvement to the previous 
patches (I believe unit tests are behaving correctly by now), and good idea to 
use {{ExecutorService}}.

Can you please address:
# {{import static}} statements ;)
# detailed and describing assertion error messages
# instead of {{Throwable#printStackTrace()}}, log message on {{ERROR}} level
# can we have different scenarios w/ more / less threads and assertion also on 
the result of {{ShareLibService#getLatestLibPath()}}?

Thanks!

> Parsing sharelib timestamps is not threadsafe
> -
>
> Key: OOZIE-3304
> URL: https://issues.apache.org/jira/browse/OOZIE-3304
> Project: Oozie
>  Issue Type: Bug
>  Components: core
>Affects Versions: 5.0.0b1, 4.3.1
>Reporter: Denes Bodo
>Assignee: Denes Bodo
>Priority: Critical
>  Labels: usability
> Attachments: OOZIE-3304-001.patch, OOZIE-3304-002.patch, 
> OOZIE-3304-003.patch, OOZIE-3304-004.patch
>
>
> In rare cases the following Exception can be read in log files when an action 
> fails:
> {code:java}
> org.apache.oozie.action.ActionExecutorException: NumberFormatException: 
> multiple points
>   at 
> org.apache.oozie.action.ActionExecutor.convertException(ActionExecutor.java:446)
>   at 
> org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:1271)
>   at 
> org.apache.oozie.action.hadoop.JavaActionExecutor.start(JavaActionExecutor.java:1472)
>   at 
> org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:234)
>   at 
> org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:65)
>   at org.apache.oozie.command.XCommand.call(XCommand.java:287)
>   at 
> org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:332)
>   at 
> org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:261)
>   at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>   at 
> org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:179)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>   at java.lang.Thread.run(Thread.java:748)
> Caused by: java.lang.NumberFormatException: multiple points
>   at 
> sun.misc.FloatingDecimal.readJavaFormatString(FloatingDecimal.java:1890)
>   at sun.misc.FloatingDecimal.parseDouble(FloatingDecimal.java:110)
>   at java.lang.Double.parseDouble(Double.java:538)
>   at java.text.DigitList.getDouble(DigitList.java:169)
>   at java.text.DecimalFormat.parse(DecimalFormat.java:2056)
>   at java.text.SimpleDateFormat.subParse(SimpleDateFormat.java:2160)
>   at java.text.SimpleDateFormat.parse(SimpleDateFormat.java:1514)
>   at java.text.DateFormat.parse(DateFormat.java:364)
>   at 
> org.apache.oozie.service.ShareLibService.getLatestLibPath(ShareLibService.java:727)
>   at 
> org.apache.oozie.service.ShareLibService.getShareLibRootPath(ShareLibService.java:595)
>   at 
> org.apache.oozie.action.hadoop.JavaActionExecutor.getSharelibRoot(JavaActionExecutor.java:1297)
>   at 
> org.apache.oozie.action.hadoop.JavaActionExecutor.initShareLibExcluder(JavaActionExecutor.java:858)
>   at 
> org.apache.oozie.action.hadoop.JavaActionExecutor.setLibFilesArchives(JavaActionExecutor.java:866)
>   at 
> org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:1156)
>   ... 11 more
> {code}
> or
> {code:java}
> 2018-07-12 04:48:52,649  WARN ForkedActionStartXCommand:523 - 
> SERVER[ctr-e138-1518143905142-410551-01-03.hwx.site] USER[user] GROUP[-] 
> TOKEN[] APP[demo-wf] JOB[023-180712043119670-oozie-oozi-W] 
> ACTION[023-180712043119670-oozie-oozi-W@streaming-node] Error starting 
> action [streaming-node]. ErrorType [ERROR], ErrorCode 
> [NumberFormatException], Message [NumberFormatException: For input string: ""]
> org.apache.oozie.action.ActionExecutorException: NumberFormatException: For 
> input string: ""
>   at 
> org.apache.oozie.action.ActionExecutor.convertException(ActionExecutor.java:446)
>   at 
> org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:1271)
>   at 
> org.apache.oozie.action.hadoop.JavaActionExecutor.start(JavaActionExecutor.java:1472)
>   at 
> org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:234)
>   at 

[jira] [Commented] (OOZIE-2877) Oozie Git Action

[Andras Piros (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16573216#comment-16573216
 ] 

Andras Piros commented on OOZIE-2877:
-

Thanks [~clayb]! I'll provide a patch soon that covers the very few steps 
remaining based on the review comments of [~pbacsko].

> Oozie Git Action
> 
>
> Key: OOZIE-2877
> URL: https://issues.apache.org/jira/browse/OOZIE-2877
> Project: Oozie
>  Issue Type: Sub-task
>  Components: action
>Reporter: Clay B.
>Assignee: Clay B.
>Priority: Major
>  Labels: action
> Fix For: trunk
>
> Attachments: 0001-OOZIE-2877-Oozie-Git-Action.patch, 
> 0002-OOZIE-2877-Oozie-Git-Action.patch, 
> 0003-OOZIE-2877-Oozie-Git-Action.patch, 
> 0004-OOZIE-2877-Oozie-Git-Action.patch, 
> 0005-OOZIE-2877-Oozie-Git-Action.patch, 
> 0006-OOZIE-2877-Oozie-Git-Action.patch, 
> 0007-OOZIE-2877-Oozie-Git-Action.patch, 
> 0008-OOZIE-2877-Oozie-Git-Action.patch, 
> 0009-OOZIE-2877-Oozie-Git-Action.patch, OOZIE-2877.010.patch, 
> OOZIE-2877.011.patch, OOZIE-2877.012.patch, OOZIE-2877.013-1.patch, 
> OOZIE-2877.013.patch, OOZIE-2877.014-1.patch, OOZIE-2877.014-2.patch, 
> OOZIE-2877.014-3.patch
>
>
> To aide in deploying ASCII artifacts to clusters, let's provide a tie-in for 
> a source-code management system. Git would be my preferred choice.
> Ideally, this could handle a user's key material e.g. for an ssh key to pull 
> down from a secured repository. This would free users from handling their own 
> key staging and clean-up on YARN nodes and only require them to store the key 
> secured in HDFS.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Review Request 67999: Parsing sharelib timestamps is not threadsafe

[Denes Bodo]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67999/
---

(Updated Aug. 8, 2018, 4:58 p.m.)


Review request for oozie and András Piros.


Bugs: OOZIE-3304
https://issues.apache.org/jira/browse/OOZIE-3304


Repository: oozie-git


Description
---

In rare cases the fNumberFormatException occurs while parsing latest sharelib 
directory names


Diffs (updated)
-

  core/src/main/java/org/apache/oozie/service/ShareLibService.java a901567 
  core/src/test/java/org/apache/oozie/service/TestHAShareLibService.java 
5087505 
  core/src/test/java/org/apache/oozie/service/TestShareLibService.java d244166 


Diff: https://reviews.apache.org/r/67999/diff/2/

Changes: https://reviews.apache.org/r/67999/diff/1-2/


Testing
---


Thanks,

Denes Bodo

[jira] [Commented] (OOZIE-3304) Parsing sharelib timestamps is not threadsafe

[Denes Bodo (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16573328#comment-16573328
 ] 

Denes Bodo commented on OOZIE-3304:
---

Diff is updated on https://reviews.apache.org/r/67999/

> Parsing sharelib timestamps is not threadsafe
> -
>
> Key: OOZIE-3304
> URL: https://issues.apache.org/jira/browse/OOZIE-3304
> Project: Oozie
>  Issue Type: Bug
>  Components: core
>Affects Versions: 5.0.0b1, 4.3.1
>Reporter: Denes Bodo
>Assignee: Denes Bodo
>Priority: Critical
>  Labels: usability
> Attachments: OOZIE-3304-001.patch, OOZIE-3304-002.patch, 
> OOZIE-3304-003.patch, OOZIE-3304-004.patch, OOZIE-3304-005.patch
>
>
> In rare cases the following Exception can be read in log files when an action 
> fails:
> {code:java}
> org.apache.oozie.action.ActionExecutorException: NumberFormatException: 
> multiple points
>   at 
> org.apache.oozie.action.ActionExecutor.convertException(ActionExecutor.java:446)
>   at 
> org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:1271)
>   at 
> org.apache.oozie.action.hadoop.JavaActionExecutor.start(JavaActionExecutor.java:1472)
>   at 
> org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:234)
>   at 
> org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:65)
>   at org.apache.oozie.command.XCommand.call(XCommand.java:287)
>   at 
> org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:332)
>   at 
> org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:261)
>   at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>   at 
> org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:179)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>   at java.lang.Thread.run(Thread.java:748)
> Caused by: java.lang.NumberFormatException: multiple points
>   at 
> sun.misc.FloatingDecimal.readJavaFormatString(FloatingDecimal.java:1890)
>   at sun.misc.FloatingDecimal.parseDouble(FloatingDecimal.java:110)
>   at java.lang.Double.parseDouble(Double.java:538)
>   at java.text.DigitList.getDouble(DigitList.java:169)
>   at java.text.DecimalFormat.parse(DecimalFormat.java:2056)
>   at java.text.SimpleDateFormat.subParse(SimpleDateFormat.java:2160)
>   at java.text.SimpleDateFormat.parse(SimpleDateFormat.java:1514)
>   at java.text.DateFormat.parse(DateFormat.java:364)
>   at 
> org.apache.oozie.service.ShareLibService.getLatestLibPath(ShareLibService.java:727)
>   at 
> org.apache.oozie.service.ShareLibService.getShareLibRootPath(ShareLibService.java:595)
>   at 
> org.apache.oozie.action.hadoop.JavaActionExecutor.getSharelibRoot(JavaActionExecutor.java:1297)
>   at 
> org.apache.oozie.action.hadoop.JavaActionExecutor.initShareLibExcluder(JavaActionExecutor.java:858)
>   at 
> org.apache.oozie.action.hadoop.JavaActionExecutor.setLibFilesArchives(JavaActionExecutor.java:866)
>   at 
> org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:1156)
>   ... 11 more
> {code}
> or
> {code:java}
> 2018-07-12 04:48:52,649  WARN ForkedActionStartXCommand:523 - 
> SERVER[ctr-e138-1518143905142-410551-01-03.hwx.site] USER[user] GROUP[-] 
> TOKEN[] APP[demo-wf] JOB[023-180712043119670-oozie-oozi-W] 
> ACTION[023-180712043119670-oozie-oozi-W@streaming-node] Error starting 
> action [streaming-node]. ErrorType [ERROR], ErrorCode 
> [NumberFormatException], Message [NumberFormatException: For input string: ""]
> org.apache.oozie.action.ActionExecutorException: NumberFormatException: For 
> input string: ""
>   at 
> org.apache.oozie.action.ActionExecutor.convertException(ActionExecutor.java:446)
>   at 
> org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:1271)
>   at 
> org.apache.oozie.action.hadoop.JavaActionExecutor.start(JavaActionExecutor.java:1472)
>   at 
> org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:234)
>   at 
> org.apache.oozie.command.wf.ForkedActionStartXCommand.execute(ForkedActionStartXCommand.java:41)
>   at 
> org.apache.oozie.command.wf.ForkedActionStartXCommand.execute(ForkedActionStartXCommand.java:30)
>   at org.apache.oozie.command.XCommand.call(XCommand.java:287)
>   at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>   at 
> org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQ

[jira] [Updated] (OOZIE-3304) Parsing sharelib timestamps is not threadsafe

[Denes Bodo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Denes Bodo updated OOZIE-3304:
--
Attachment: OOZIE-3304-005.patch

> Parsing sharelib timestamps is not threadsafe
> -
>
> Key: OOZIE-3304
> URL: https://issues.apache.org/jira/browse/OOZIE-3304
> Project: Oozie
>  Issue Type: Bug
>  Components: core
>Affects Versions: 5.0.0b1, 4.3.1
>Reporter: Denes Bodo
>Assignee: Denes Bodo
>Priority: Critical
>  Labels: usability
> Attachments: OOZIE-3304-001.patch, OOZIE-3304-002.patch, 
> OOZIE-3304-003.patch, OOZIE-3304-004.patch, OOZIE-3304-005.patch
>
>
> In rare cases the following Exception can be read in log files when an action 
> fails:
> {code:java}
> org.apache.oozie.action.ActionExecutorException: NumberFormatException: 
> multiple points
>   at 
> org.apache.oozie.action.ActionExecutor.convertException(ActionExecutor.java:446)
>   at 
> org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:1271)
>   at 
> org.apache.oozie.action.hadoop.JavaActionExecutor.start(JavaActionExecutor.java:1472)
>   at 
> org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:234)
>   at 
> org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:65)
>   at org.apache.oozie.command.XCommand.call(XCommand.java:287)
>   at 
> org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:332)
>   at 
> org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:261)
>   at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>   at 
> org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:179)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>   at java.lang.Thread.run(Thread.java:748)
> Caused by: java.lang.NumberFormatException: multiple points
>   at 
> sun.misc.FloatingDecimal.readJavaFormatString(FloatingDecimal.java:1890)
>   at sun.misc.FloatingDecimal.parseDouble(FloatingDecimal.java:110)
>   at java.lang.Double.parseDouble(Double.java:538)
>   at java.text.DigitList.getDouble(DigitList.java:169)
>   at java.text.DecimalFormat.parse(DecimalFormat.java:2056)
>   at java.text.SimpleDateFormat.subParse(SimpleDateFormat.java:2160)
>   at java.text.SimpleDateFormat.parse(SimpleDateFormat.java:1514)
>   at java.text.DateFormat.parse(DateFormat.java:364)
>   at 
> org.apache.oozie.service.ShareLibService.getLatestLibPath(ShareLibService.java:727)
>   at 
> org.apache.oozie.service.ShareLibService.getShareLibRootPath(ShareLibService.java:595)
>   at 
> org.apache.oozie.action.hadoop.JavaActionExecutor.getSharelibRoot(JavaActionExecutor.java:1297)
>   at 
> org.apache.oozie.action.hadoop.JavaActionExecutor.initShareLibExcluder(JavaActionExecutor.java:858)
>   at 
> org.apache.oozie.action.hadoop.JavaActionExecutor.setLibFilesArchives(JavaActionExecutor.java:866)
>   at 
> org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:1156)
>   ... 11 more
> {code}
> or
> {code:java}
> 2018-07-12 04:48:52,649  WARN ForkedActionStartXCommand:523 - 
> SERVER[ctr-e138-1518143905142-410551-01-03.hwx.site] USER[user] GROUP[-] 
> TOKEN[] APP[demo-wf] JOB[023-180712043119670-oozie-oozi-W] 
> ACTION[023-180712043119670-oozie-oozi-W@streaming-node] Error starting 
> action [streaming-node]. ErrorType [ERROR], ErrorCode 
> [NumberFormatException], Message [NumberFormatException: For input string: ""]
> org.apache.oozie.action.ActionExecutorException: NumberFormatException: For 
> input string: ""
>   at 
> org.apache.oozie.action.ActionExecutor.convertException(ActionExecutor.java:446)
>   at 
> org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:1271)
>   at 
> org.apache.oozie.action.hadoop.JavaActionExecutor.start(JavaActionExecutor.java:1472)
>   at 
> org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:234)
>   at 
> org.apache.oozie.command.wf.ForkedActionStartXCommand.execute(ForkedActionStartXCommand.java:41)
>   at 
> org.apache.oozie.command.wf.ForkedActionStartXCommand.execute(ForkedActionStartXCommand.java:30)
>   at org.apache.oozie.command.XCommand.call(XCommand.java:287)
>   at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>   at 
> org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:179)
>   at 
> java.util.concurrent.ThreadPoolExecutor.

[jira] [Commented] (OOZIE-3304) Parsing sharelib timestamps is not threadsafe

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16573334#comment-16573334
 ] 

Hadoop QA commented on OOZIE-3304:
--

PreCommit-OOZIE-Build started


> Parsing sharelib timestamps is not threadsafe
> -
>
> Key: OOZIE-3304
> URL: https://issues.apache.org/jira/browse/OOZIE-3304
> Project: Oozie
>  Issue Type: Bug
>  Components: core
>Affects Versions: 5.0.0b1, 4.3.1
>Reporter: Denes Bodo
>Assignee: Denes Bodo
>Priority: Critical
>  Labels: usability
> Attachments: OOZIE-3304-001.patch, OOZIE-3304-002.patch, 
> OOZIE-3304-003.patch, OOZIE-3304-004.patch, OOZIE-3304-005.patch
>
>
> In rare cases the following Exception can be read in log files when an action 
> fails:
> {code:java}
> org.apache.oozie.action.ActionExecutorException: NumberFormatException: 
> multiple points
>   at 
> org.apache.oozie.action.ActionExecutor.convertException(ActionExecutor.java:446)
>   at 
> org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:1271)
>   at 
> org.apache.oozie.action.hadoop.JavaActionExecutor.start(JavaActionExecutor.java:1472)
>   at 
> org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:234)
>   at 
> org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:65)
>   at org.apache.oozie.command.XCommand.call(XCommand.java:287)
>   at 
> org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:332)
>   at 
> org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:261)
>   at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>   at 
> org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:179)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>   at java.lang.Thread.run(Thread.java:748)
> Caused by: java.lang.NumberFormatException: multiple points
>   at 
> sun.misc.FloatingDecimal.readJavaFormatString(FloatingDecimal.java:1890)
>   at sun.misc.FloatingDecimal.parseDouble(FloatingDecimal.java:110)
>   at java.lang.Double.parseDouble(Double.java:538)
>   at java.text.DigitList.getDouble(DigitList.java:169)
>   at java.text.DecimalFormat.parse(DecimalFormat.java:2056)
>   at java.text.SimpleDateFormat.subParse(SimpleDateFormat.java:2160)
>   at java.text.SimpleDateFormat.parse(SimpleDateFormat.java:1514)
>   at java.text.DateFormat.parse(DateFormat.java:364)
>   at 
> org.apache.oozie.service.ShareLibService.getLatestLibPath(ShareLibService.java:727)
>   at 
> org.apache.oozie.service.ShareLibService.getShareLibRootPath(ShareLibService.java:595)
>   at 
> org.apache.oozie.action.hadoop.JavaActionExecutor.getSharelibRoot(JavaActionExecutor.java:1297)
>   at 
> org.apache.oozie.action.hadoop.JavaActionExecutor.initShareLibExcluder(JavaActionExecutor.java:858)
>   at 
> org.apache.oozie.action.hadoop.JavaActionExecutor.setLibFilesArchives(JavaActionExecutor.java:866)
>   at 
> org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:1156)
>   ... 11 more
> {code}
> or
> {code:java}
> 2018-07-12 04:48:52,649  WARN ForkedActionStartXCommand:523 - 
> SERVER[ctr-e138-1518143905142-410551-01-03.hwx.site] USER[user] GROUP[-] 
> TOKEN[] APP[demo-wf] JOB[023-180712043119670-oozie-oozi-W] 
> ACTION[023-180712043119670-oozie-oozi-W@streaming-node] Error starting 
> action [streaming-node]. ErrorType [ERROR], ErrorCode 
> [NumberFormatException], Message [NumberFormatException: For input string: ""]
> org.apache.oozie.action.ActionExecutorException: NumberFormatException: For 
> input string: ""
>   at 
> org.apache.oozie.action.ActionExecutor.convertException(ActionExecutor.java:446)
>   at 
> org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:1271)
>   at 
> org.apache.oozie.action.hadoop.JavaActionExecutor.start(JavaActionExecutor.java:1472)
>   at 
> org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:234)
>   at 
> org.apache.oozie.command.wf.ForkedActionStartXCommand.execute(ForkedActionStartXCommand.java:41)
>   at 
> org.apache.oozie.command.wf.ForkedActionStartXCommand.execute(ForkedActionStartXCommand.java:30)
>   at org.apache.oozie.command.XCommand.call(XCommand.java:287)
>   at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>   at 
> org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:179)
>   

Failed: OOZIE-3304 PreCommit Build #739

[Apache Jenkins Server]

Jira: https://issues.apache.org/jira/browse/OOZIE-3304
Build: https://builds.apache.org/job/PreCommit-OOZIE-Build/739/

###
## LAST 100 LINES OF THE CONSOLE 
###
[...truncated 454.03 KB...]
[INFO] Apache Oozie Docs .. SUCCESS [ 10.322 s]
[INFO] Apache Oozie WebApp  SUCCESS [ 31.916 s]
[INFO] Apache Oozie Tools . SUCCESS [  1.369 s]
[INFO] Apache Oozie MiniOozie . SUCCESS [  1.365 s]
[INFO] Apache Oozie Fluent Job Client . SUCCESS [  0.240 s]
[INFO] Apache Oozie Server  SUCCESS [  3.575 s]
[INFO] Apache Oozie Distro  SUCCESS [01:00 min]
[INFO] Apache Oozie ZooKeeper Security Tests 5.1.0-SNAPSHOT SUCCESS [  2.456 s]
[INFO] 
[INFO] BUILD SUCCESS
[INFO] 
[INFO] Total time: 03:36 min
[INFO] Finished at: 2018-08-08T16:33:15Z
[INFO] 

Reports
  Running test-patch task CLEAN
  Running test-patch task RAW_PATCH_ANALYSIS
  Running test-patch task RAT
  Running test-patch task JAVADOC
  Running test-patch task COMPILE
  Running test-patch task FINDBUGS_DIFF
[TRACE] Downloading FindBugs diff JAR from 
https://repo1.maven.org/maven2/me/andrz/findbugs/findbugs-diff/0.1.0/findbugs-diff-0.1.0-all.jar
[TRACE] FindBugs diff JAR downloaded
grep: 
/home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/bin/findbugs-diff-0.1.0-all.jar.md5sum:
 No such file or directory
[TRACE] FindBugs diff JAR checked, is safe to use
[TRACE] Performing FindBugs diffs
[TRACE] FindBugs diffs performed
[TRACE] Checking FindBugs diffs and creating reports
[INFO] There are no new bugs found totally].
[TRACE] FindBugs diffs checked and reports created
[TRACE] Summary file size is 1233 bytes
[TRACE] Full summary file size is 184 bytes
[TRACE] File 
[/home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/tmp/FINDBUGS_DIFF/diff/findbugs-diff-0.1.0-all.jar]
 removed
[TRACE] File 
[/home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/tmp/FINDBUGS_DIFF/diff/findbugs-diff-0.1.0-all.jar.md5sum]
 removed
  Running test-patch task BACKWARDS_COMPATIBILITY
  Running test-patch task TESTS
  Running test-patch task DISTRO


Testing JIRA OOZIE-3304

Cleaning local git workspace



+1 PATCH_APPLIES
+1 CLEAN
-1 RAW_PATCH_ANALYSIS
+1 the patch does not introduce any @author tags
+1 the patch does not introduce any tabs
+1 the patch does not introduce any trailing spaces
-1 the patch contains 1 line(s) longer than 132 characters
+1 the patch adds/modifies 2 testcase(s)
+1 RAT
+1 the patch does not seem to introduce new RAT warnings
+1 JAVADOC
+1 JAVADOC
+1 the patch does not seem to introduce new Javadoc warning(s)
+1 the patch does not seem to introduce new Javadoc error(s)
ERROR: the current HEAD has 2 Javadoc error(s)
+1 COMPILE
+1 HEAD compiles
+1 patch compiles
+1 the patch does not seem to introduce new javac warnings
+1 There are no new bugs found in total.
+1 BACKWARDS_COMPATIBILITY
+1 the patch does not change any JPA Entity/Colum/Basic/Lob/Transient 
annotations
+1 the patch does not modify JPA files
+1 TESTS
Tests run: 2918
Tests failed at first run:
TestCoordActionsKillXCommand#testActionKillCommandDate
For the complete list of flaky tests, see TEST-SUMMARY-FULL files.
+1 DISTRO
+1 distro tarball builds with the patch 


-1 Overall result, please check the reported -1(s)


The full output of the test-patch run is available at

 https://builds.apache.org/job/PreCommit-OOZIE-Build/739/

Adding comment to JIRA
  % Total% Received % Xferd  Average Speed   TimeTime Time  Current
 Dload  Upload   Total   SpentLeft  Speed
  0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 
0100  54820  3429  100  2053   3977   2381 --:--:-- --:--:-- --:--:--  3982
{"self":"https://issues.apache.org/jira/rest/api/2/issue/13172628/comment/16573462","id":"16573462","author":{"self":"https://issues.apache.org/jira/rest/api/2/user?username=hadoopqa","name":"hadoopqa","key":"hadoopqa","avatarUrls":{"48x48":"https://issues.apache.org/jira/secure/useravatar?ownerId=hadoopqa&avatarId=10393","24x24":"https://issues.apache.org/jira/secure/useravatar?size=small&ownerId=hadoopqa&avatarId=10393","16x16":"https://issues.apache.org/jira/secure/useravatar?size=xsmall&ownerId=hadoopqa&avatarId=10393","32x32":"https://issues.apache.org/jira/secure/useravatar?size=medium&ownerId=hadoopqa&avatarId=10393"},"displayName"

[jira] [Commented] (OOZIE-3304) Parsing sharelib timestamps is not threadsafe

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16573462#comment-16573462
 ] 

Hadoop QA commented on OOZIE-3304:
--


Testing JIRA OOZIE-3304

Cleaning local git workspace



{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:red}-1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:red}-1{color} the patch contains 1 line(s) longer than 132 
characters
.{color:green}+1{color} the patch adds/modifies 2 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT 
warnings
{color:green}+1 JAVADOC{color}
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc 
warning(s)
.{color:green}+1{color} the patch does not seem to introduce new Javadoc 
error(s)
.{color:red}ERROR{color}: the current HEAD has 2 Javadoc error(s)
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac 
warnings
{color:green}+1{color} There are no new bugs found in total.
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA 
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:green}+1 TESTS{color}
.Tests run: 2918
.{color:orange}Tests failed at first run:{color}
TestCoordActionsKillXCommand#testActionKillCommandDate
.For the complete list of flaky tests, see TEST-SUMMARY-FULL files.
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch 


{color:red}*-1 Overall result, please check the reported -1(s)*{color}


The full output of the test-patch run is available at

. https://builds.apache.org/job/PreCommit-OOZIE-Build/739/



> Parsing sharelib timestamps is not threadsafe
> -
>
> Key: OOZIE-3304
> URL: https://issues.apache.org/jira/browse/OOZIE-3304
> Project: Oozie
>  Issue Type: Bug
>  Components: core
>Affects Versions: 5.0.0b1, 4.3.1
>Reporter: Denes Bodo
>Assignee: Denes Bodo
>Priority: Critical
>  Labels: usability
> Attachments: OOZIE-3304-001.patch, OOZIE-3304-002.patch, 
> OOZIE-3304-003.patch, OOZIE-3304-004.patch, OOZIE-3304-005.patch
>
>
> In rare cases the following Exception can be read in log files when an action 
> fails:
> {code:java}
> org.apache.oozie.action.ActionExecutorException: NumberFormatException: 
> multiple points
>   at 
> org.apache.oozie.action.ActionExecutor.convertException(ActionExecutor.java:446)
>   at 
> org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:1271)
>   at 
> org.apache.oozie.action.hadoop.JavaActionExecutor.start(JavaActionExecutor.java:1472)
>   at 
> org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:234)
>   at 
> org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:65)
>   at org.apache.oozie.command.XCommand.call(XCommand.java:287)
>   at 
> org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:332)
>   at 
> org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:261)
>   at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>   at 
> org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:179)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>   at java.lang.Thread.run(Thread.java:748)
> Caused by: java.lang.NumberFormatException: multiple points
>   at 
> sun.misc.FloatingDecimal.readJavaFormatString(FloatingDecimal.java:1890)
>   at sun.misc.FloatingDecimal.parseDouble(FloatingDecimal.java:110)
>   at java.lang.Double.parseDouble(Double.java:538)
>   at java.text.DigitList.getDouble(DigitList.java:169)
>   at java.text.DecimalFormat.parse(DecimalFormat.java:2056)
>   at java.text.SimpleDateFormat.subParse(SimpleDateFormat.java:2160)
>   at java.text.SimpleDateFormat.parse(SimpleDateFormat.java:1514)
>   at java.text.DateFormat.parse(DateFormat.java:364)
>   at 
> org.apache.oozie.service.ShareLibService.getLatestLibPath(ShareLibService.java:727)
>   at 
> org.apache.oo

[jira] Subscription: Oozie Patch Available

[jira]

Issue Subscription
Filter: Oozie Patch Available (95 issues)

Subscriber: ooziedaily

Key Summary
OOZIE-3320  Oozie ShellAction should support absolute bash file path
https://issues.apache.org/jira/browse/OOZIE-3320
OOZIE-3319  Log SSH action callback error output
https://issues.apache.org/jira/browse/OOZIE-3319
OOZIE-3318  Fix Javadoc check in the pre-commit
https://issues.apache.org/jira/browse/OOZIE-3318
OOZIE-3304  Parsing sharelib timestamps is not threadsafe
https://issues.apache.org/jira/browse/OOZIE-3304
OOZIE-3301  Update NOTICE file
https://issues.apache.org/jira/browse/OOZIE-3301
OOZIE-3298  OYA: external ID is not filled properly and failing MR job is 
treated as SUCCEEDED
https://issues.apache.org/jira/browse/OOZIE-3298
OOZIE-3274  Remove slf4j
https://issues.apache.org/jira/browse/OOZIE-3274
OOZIE-3266  Coord action rerun support RERUN_SKIP_NODES option
https://issues.apache.org/jira/browse/OOZIE-3266
OOZIE-3265  properties RERUN_FAIL_NODES and RERUN_SKIP_NODES should be able to 
appear together
https://issues.apache.org/jira/browse/OOZIE-3265
OOZIE-3256  refactor OozieCLI class
https://issues.apache.org/jira/browse/OOZIE-3256
OOZIE-3249  [tools] Instrumentation log parser
https://issues.apache.org/jira/browse/OOZIE-3249
OOZIE-3229  Improved filtering options in V2SLAServlet
https://issues.apache.org/jira/browse/OOZIE-3229
OOZIE-3218  Oozie Sqoop action with command splits the select clause into 
multiple parts due to delimiter being space
https://issues.apache.org/jira/browse/OOZIE-3218
OOZIE-3199  Let system property restriction configurable
https://issues.apache.org/jira/browse/OOZIE-3199
OOZIE-3196  Authorization: restrict world readability by user
https://issues.apache.org/jira/browse/OOZIE-3196
OOZIE-3194  Oozie should set proper permissions to sharelib after upload
https://issues.apache.org/jira/browse/OOZIE-3194
OOZIE-3186  Oozie is unable to use configuration linked using jceks://file/...
https://issues.apache.org/jira/browse/OOZIE-3186
OOZIE-3179  Adding a configurable config-default.xml location to a workflow
https://issues.apache.org/jira/browse/OOZIE-3179
OOZIE-3170  Oozie Diagnostic Bundle tool fails with NPE due to missing service 
class
https://issues.apache.org/jira/browse/OOZIE-3170
OOZIE-3160  PriorityDelayQueue put()/take() can cause significant CPU load due 
to busy waiting
https://issues.apache.org/jira/browse/OOZIE-3160
OOZIE-3135  Configure log4j2 in SqoopMain
https://issues.apache.org/jira/browse/OOZIE-3135
OOZIE-3091  Oozie Sqoop Avro Import fails with "java.lang.NoClassDefFoundError: 
org/apache/avro/mapred/AvroWrapper"
https://issues.apache.org/jira/browse/OOZIE-3091
OOZIE-3071  Oozie 4.3 Spark sharelib ueses a different version of commons-lang3 
than Spark 2.2.0
https://issues.apache.org/jira/browse/OOZIE-3071
OOZIE-3063  Sanitizing variables that are part of openjpa.ConnectionProperties
https://issues.apache.org/jira/browse/OOZIE-3063
OOZIE-3062  Set HADOOP_CONF_DIR for spark action
https://issues.apache.org/jira/browse/OOZIE-3062
OOZIE-3061  Kill only those child jobs which are not already killed
https://issues.apache.org/jira/browse/OOZIE-3061
OOZIE-2952  Fix Findbugs warnings in oozie-sharelib-oozie
https://issues.apache.org/jira/browse/OOZIE-2952
OOZIE-2949  Escape quotes whitespaces in Sqoop  field
https://issues.apache.org/jira/browse/OOZIE-2949
OOZIE-2927  Append new line character for Hive2 query using query tag
https://issues.apache.org/jira/browse/OOZIE-2927
OOZIE-2877  Oozie Git Action
https://issues.apache.org/jira/browse/OOZIE-2877
OOZIE-2834  ParameterVerifier logging non-useful warning for workflow definition
https://issues.apache.org/jira/browse/OOZIE-2834
OOZIE-2833  when using uber mode the regex pattern used in the 
extractHeapSizeMB method does not allow heap sizes specified in bytes.
https://issues.apache.org/jira/browse/OOZIE-2833
OOZIE-2812  SparkConfigurationService should support loading configurations 
from multiple Spark versions
https://issues.apache.org/jira/browse/OOZIE-2812
OOZIE-2795  Create lib directory or symlink for Oozie CLI during packaging
https://issues.apache.org/jira/browse/OOZIE-2795
OOZIE-2784  Include WEEK as a parameter in the Coordinator Expression Language 
Evaulator
https://issues.apache.org/jira/browse/OOZIE-2784
OOZIE-2779  Mask Hive2 action Beeline JDBC password
https://issues.apache.org/jira/browse/OOZIE-2779
OOZIE-2736  Reduce the number of threads during test execution
https://issues.apache.org/jira/browse/OOZIE-2736
OOZIE-2734  Switch docs from twiki to markdown
  

[jira] [Commented] (OOZIE-3320) Oozie ShellAction should support absolute bash file path

[Attila Sasvari (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-3320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16574357#comment-16574357
 ] 

Attila Sasvari commented on OOZIE-3320:
---

Thanks [~roeyshemtov24]. It duplicates 
[OOZIE-3086|https://issues.apache.org/jira/browse/OOZIE-3086].

> Oozie ShellAction should support absolute bash file path
> 
>
> Key: OOZIE-3320
> URL: https://issues.apache.org/jira/browse/OOZIE-3320
> Project: Oozie
>  Issue Type: Improvement
>  Components: action
>Affects Versions: 5.0.0, 4.3.1
>Reporter: Roey Shem Tov
>Assignee: Roey Shem Tov
>Priority: Major
>  Labels: patch
> Attachments: OOZIE-3086.patch
>
>
> bash files that saved on shared mount, cannot be execute by the ShellAction 
> in proper way.
> Example:
> Worker-1,Worker-2,Worker-3 have shared mount /mnt/hadoop
> on /mnt/hadoop there is a file script.sh
> Right now there is two options to submit it using ShellAction:
>  # Upload it to hdfs, add it as a file and submit script.sh
>  # use bash as exec and file location (/mnt/hadoop/script.sh) as argument 
> (e.g bash>/exec>/mnt/hadoop/script.sh
> Best option is that the  command will support shared mounted file :
> /mnt/hadoop/script.sh
>  
> This code is taking only the file name instead it full path:
> {code:java}
> String exec = actionXml.getChild("exec", ns).getTextTrim(); 
> String execName = new Path(exec).getName(); 
> actionConf.set(ShellMain.CONF_OOZIE_SHELL_EXEC, execName);
> {code}
>  
> Best option to support shared mount file is to support file:// starting for 
> bash files that are local (or shared by mount), e.g:
> {code:java}
> String exec = actionXml.getChild("exec", ns).getTextTrim(); String execName; 
> String localFilePrefix = "file://";
>  // When exec starts with 'file://' refer it as local file. 
> if (exec.startsWith(localFilePrefix)) 
> execName = exec.substring(localFilePrefix.length()); 
> else execName = new Path(exec).getName(); 
> actionConf.set(ShellMain.CONF_OOZIE_SHELL_EXEC, execName);
> {code}
>  
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Resolved] (OOZIE-3086) Oozie shell action ignores absolute path

[Attila Sasvari (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-3086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Attila Sasvari resolved OOZIE-3086.
---
Resolution: Duplicate

> Oozie shell action ignores absolute path
> 
>
> Key: OOZIE-3086
> URL: https://issues.apache.org/jira/browse/OOZIE-3086
> Project: Oozie
>  Issue Type: Bug
>  Components: action
>Affects Versions: 3.1.3, 3.2.0, 3.3.0, trunk, 3.3.1, 3.3.2, 4.0.0, 4.1.0, 
> 4.0.1, 4.2.0, 4.3.0
>Reporter: Attila Sasvari
>Priority: Major
>
> [ShellActionExecutor's setupActionConf() method| 
> https://github.com/apache/oozie/blob/883c7556d2f302261eae5aec6e323a5b1de04f74/core/src/main/java/org/apache/oozie/action/hadoop/ShellActionExecutor.java#L57]
>  retrieves only the *basename* for the shell script specified by the user.
> {code}
> Configuration setupActionConf(Configuration actionConf, Context context, 
> Element actionXml, Path appPath)
> {
> ...
> String exec = actionXml.getChild("exec", ns).getTextTrim();
> String execName = new Path(exec).getName();
> actionConf.set(ShellMain.CONF_OOZIE_SHELL_EXEC, execName);
> {code}
> As a result, {{ShellMain}} will 
> [execute|https://github.com/apache/oozie/blob/883c7556d2f302261eae5aec6e323a5b1de04f74/sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/ShellMain.java#L356]
>  the default command available in $PATH.
> {code}
> protected String getExec(Configuration actionConf) {
> String exec = actionConf.get(CONF_OOZIE_SHELL_EXEC);
> {code}
> Consequences:
> - This behaviour makes it also hard/impossible to use custom executable at a 
> given location (that is available on each node where a shell action can run, 
> for example a specific python version).
> - Even if a user specifies a wrong path (e.g. a non-existing {{/usr/bn/id}}) 
> on purpose to force failure of a shell action, the command might succeed with 
> no apparent reason.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)