[jira] [Updated] (OOZIE-2244) Oozie should mask passwords in the logs when logging command arguments

[Venkat Ranganathan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-2244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Venkat Ranganathan updated OOZIE-2244:
--
Attachment: OOZIE-2244-01.patch

Rebased to trunk


> Oozie should mask passwords in the logs when logging command arguments
> --
>
> Key: OOZIE-2244
> URL: https://issues.apache.org/jira/browse/OOZIE-2244
> Project: Oozie
>  Issue Type: Bug
>Affects Versions: 4.1.0, 4.0.1
> Environment: All
>Reporter: Venkat Ranganathan
>Assignee: Venkat Ranganathan
>Priority: Critical
> Fix For: trunk
>
> Attachments: OOZIE-2244-01.patch, OOZIE-2244-no-prefix.patch
>
>
> Users have complained that oozie logging the password related argument values 
> in the launcher log is a security hole and want it to be masked in the 
> output.   Even password aliases in keystore are considered to be a security 
> hole.
> The fix is to mask any argument values if option name contains the string 
> password (which is true for Sqoop).   We do this in multiple places, in Sqoop 
> main, in Launcher Mapper, in JavaMain as well.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2349) Method getCoordJobInfo(String jobId, String filter, int offset, int length, boolean desc) is not present in LocalOozieClientCoord

[Narayan Periwal (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15405372#comment-15405372
 ] 

Narayan Periwal commented on OOZIE-2349:


Thanks [~rkanter]

> Method getCoordJobInfo(String jobId, String filter, int offset, int length, 
> boolean desc) is not present in LocalOozieClientCoord
> -
>
> Key: OOZIE-2349
> URL: https://issues.apache.org/jira/browse/OOZIE-2349
> Project: Oozie
>  Issue Type: Improvement
>Reporter: Narayan Periwal
>Assignee: Narayan Periwal
> Fix For: trunk
>
> Attachments: OOZIE-2349-v0.patch
>
>
> In LocalOozieClientCoord, method getCoordJobInfo(String jobId) is present, 
> but the corresponding overloaded method getCoordJobInfo(String jobId, String 
> filter, int offset, int length, boolean desc) is not present. Adding this 
> method in LocalOozieClientCoord will enable us to use this in the unit 
> testcases. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OOZIE-2496) Testcase failure due to environment specific line.separator value

[abhishek bafna (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-2496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

abhishek bafna updated OOZIE-2496:
--
Attachment: OOZIE-2496-03.patch

> Testcase failure due to environment specific line.separator value
> -
>
> Key: OOZIE-2496
> URL: https://issues.apache.org/jira/browse/OOZIE-2496
> Project: Oozie
>  Issue Type: Bug
>  Components: tests
> Environment: Windows
>Reporter: abhishek bafna
>Assignee: abhishek bafna
>  Labels: patch
> Attachments: OOZIE-2496-01.patch, OOZIE-2496-02.patch, 
> OOZIE-2496-03.patch
>
>
> The "line.separator" returns operating system specific line separator. The 
> following testcase uses hard coded "\n" while comparing the return values in 
> the assertion, which makes them to fail on the windows environment.
> {noformat}
> org.apache.oozie.client.TestOozieCLI#testOozieStatus
> org.apache.oozie.client.TestOozieCLI#testServerBuildVersion
> org.apache.oozie.client.TestOozieCLI#testClientBuildVersion
> {noformat}
> In the following testcase, we are using "line.separator" where as 
> _TimestampedMessageParser_ sends the logs after appending "\n".
> {noformat}
> org.apache.oozie.util.TestXLogUserFilterParam#testloglevel_DEBUF_INFO
> org.apache.oozie.util.TestXLogUserFilterParam#testloglevel_Error
> org.apache.oozie.util.TestXLogUserFilterParam#testTextSearch
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2496) Testcase failure due to environment specific line.separator value

[abhishek bafna (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15405338#comment-15405338
 ] 

abhishek bafna commented on OOZIE-2496:
---

[~rkanter] Thanks for reviewing. Incorporated the suggestions. Also added a 
{{SYSTEM_LINE_SEPARATOR}} constant into the {{XTestCase}} class, so that any 
test class can reference it in future.

> Testcase failure due to environment specific line.separator value
> -
>
> Key: OOZIE-2496
> URL: https://issues.apache.org/jira/browse/OOZIE-2496
> Project: Oozie
>  Issue Type: Bug
>  Components: tests
> Environment: Windows
>Reporter: abhishek bafna
>Assignee: abhishek bafna
>  Labels: patch
> Attachments: OOZIE-2496-01.patch, OOZIE-2496-02.patch
>
>
> The "line.separator" returns operating system specific line separator. The 
> following testcase uses hard coded "\n" while comparing the return values in 
> the assertion, which makes them to fail on the windows environment.
> {noformat}
> org.apache.oozie.client.TestOozieCLI#testOozieStatus
> org.apache.oozie.client.TestOozieCLI#testServerBuildVersion
> org.apache.oozie.client.TestOozieCLI#testClientBuildVersion
> {noformat}
> In the following testcase, we are using "line.separator" where as 
> _TimestampedMessageParser_ sends the logs after appending "\n".
> {noformat}
> org.apache.oozie.util.TestXLogUserFilterParam#testloglevel_DEBUF_INFO
> org.apache.oozie.util.TestXLogUserFilterParam#testloglevel_Error
> org.apache.oozie.util.TestXLogUserFilterParam#testTextSearch
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OOZIE-2493) TestDiscp leaves action.xml behind on local filesystem

[abhishek bafna (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-2493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

abhishek bafna updated OOZIE-2493:
--
Attachment: OOZIE-2493-03.patch

> TestDiscp leaves action.xml behind on local filesystem
> --
>
> Key: OOZIE-2493
> URL: https://issues.apache.org/jira/browse/OOZIE-2493
> Project: Oozie
>  Issue Type: Bug
>  Components: tests
>Reporter: abhishek bafna
>Assignee: abhishek bafna
>  Labels: patch
> Fix For: 4.3.0
>
> Attachments: OOZIE-2493-02.patch, OOZIE-2493-03.patch, 
> OOZIE-2493.patch
>
>
> In the TestDistcpMain#testMain test case, it is deleting a file using HDFS 
> FileSystem instance which actually created on a local FileSystem.
> {code}
> // getting file hadoop system instance 
> FileSystem fs = getFileSystem();
> ...
> // creating a storing file on local file system
> File actionXml = new File(getTestCaseDir(), "action.xml");
> ...
> // Now trying to delete the file on hadoop file system
> fs.delete(new Path(getTestCaseDir(), "action.xml"), true);
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2493) TestDiscp leaves action.xml behind on local filesystem

[abhishek bafna (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15405312#comment-15405312
 ] 

abhishek bafna commented on OOZIE-2493:
---

[~rkanter] Thanks for the review. Yes, I found the reason. After deleting the 
file we were writing into it again. made the required changes.

> TestDiscp leaves action.xml behind on local filesystem
> --
>
> Key: OOZIE-2493
> URL: https://issues.apache.org/jira/browse/OOZIE-2493
> Project: Oozie
>  Issue Type: Bug
>  Components: tests
>Reporter: abhishek bafna
>Assignee: abhishek bafna
>  Labels: patch
> Fix For: 4.3.0
>
> Attachments: OOZIE-2493-02.patch, OOZIE-2493.patch
>
>
> In the TestDistcpMain#testMain test case, it is deleting a file using HDFS 
> FileSystem instance which actually created on a local FileSystem.
> {code}
> // getting file hadoop system instance 
> FileSystem fs = getFileSystem();
> ...
> // creating a storing file on local file system
> File actionXml = new File(getTestCaseDir(), "action.xml");
> ...
> // Now trying to delete the file on hadoop file system
> fs.delete(new Path(getTestCaseDir(), "action.xml"), true);
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2516) Update web service documentation for jobs API

[abhishek bafna (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15405263#comment-15405263
 ] 

abhishek bafna commented on OOZIE-2516:
---

[~rkanter] Thanks for the review and committing to the master.

> Update web service documentation for jobs API
> -
>
> Key: OOZIE-2516
> URL: https://issues.apache.org/jira/browse/OOZIE-2516
> Project: Oozie
>  Issue Type: Bug
>  Components: docs
>Reporter: abhishek bafna
>Assignee: abhishek bafna
> Fix For: trunk
>
> Attachments: OOZIE-2516-01.patch
>
>
> 1. Support for the 'group' filter is removed from the jobs API but the 
> WebService documentation is not updated for the same.
> 2. jobs API supports the 'id' filter which is also not documented in the web 
> services.
> https://oozie.apache.org/docs/4.2.0/WebServicesAPI.html#Jobs_Information
> From WorkflowsJobGetJPAExecutor and StoreStatusFilter
> {code}
> if (entry.getKey().equals(OozieClient.FILTER_GROUP)) {
>   XLog.getLog(getClass()).warn("Filter by 'group' is not supported anymore");
> }
> ...
> if (entry.getKey().equals(OozieClient.FILTER_ID)) {
> ...
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2555) Oozie SSL enable setup does not return port for admin -servers

[abhishek bafna (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15405262#comment-15405262
 ] 

abhishek bafna commented on OOZIE-2555:
---

[~rkanter] Thanks for the review and committing to the master.

> Oozie SSL enable setup does not return port for admin -servers
> --
>
> Key: OOZIE-2555
> URL: https://issues.apache.org/jira/browse/OOZIE-2555
> Project: Oozie
>  Issue Type: Bug
>  Components: core
>Reporter: abhishek bafna
>Assignee: abhishek bafna
> Fix For: trunk
>
> Attachments: OOZIE-2555-00.patch
>
>
> When oozie is setup with SSL. The *oozie admin -servers -oozie 
> https://hostname:11443/oozie* is not returning the port in output.
> {noformat}
> [oozie@host oozie]$ oozie admin -servers -oozie https://hostname:11443/oozie
> hostname : https://hostname:/oozie
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2497) Some tests fail on windows due to hard coded URIs

[abhishek bafna (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15405257#comment-15405257
 ] 

abhishek bafna commented on OOZIE-2497:
---

[~rkanter] Thanks for the review and committing to the master.

> Some tests fail on windows due to hard coded URIs
> -
>
> Key: OOZIE-2497
> URL: https://issues.apache.org/jira/browse/OOZIE-2497
> Project: Oozie
>  Issue Type: Bug
>  Components: tests
> Environment: Windows
>Reporter: abhishek bafna
>Assignee: abhishek bafna
>  Labels: patch
> Fix For: trunk
>
> Attachments: OOZIE-2497-01.patch
>
>
> Some of the test cases have hard coded file URIs like "file://" which makes 
> them to fail in the windows environment.
> testcase:
> {noformat}
> org.apache.oozie.command.coord.TestCoordSubmitXCommand#testELdataIO_xsd_4
> org.apache.oozie.action.hadoop.TestMapReduceActionExecutor#testConfigDefaultPropsToAction
> org.apache.oozie.command.wf.TestSignalXCommand#testJoinFail
> {noformat}
> Exception message
> {noformat}
> testELdataIO_xsd_4
> java.net.URISyntaxException: Illegal character in authority at index 7: 
> file://D:\w\oozie\core\target\test-data\oozietests\org.apache.oozie.command.coord.TestCoordSubmitXCommand\testELdataIO_xsd_4\coordinator.xml
>   at java.net.URI$Parser.fail(URI.java:2827)
>   at java.net.URI$Parser.parseAuthority(URI.java:3165)
>   at java.net.URI$Parser.parseHierarchical(URI.java:3076)
>   at java.net.URI$Parser.parse(URI.java:3032)
>   at java.net.URI.(URI.java:595)
>   at 
> org.apache.oozie.command.coord.TestCoordSubmitXCommand.testELdataIO_xsd_4(TestCoordSubmitXCommand.java:576)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:601)
>   at junit.framework.TestCase.runTest(TestCase.java:168)
>   at junit.framework.TestCase.runBare(TestCase.java:134)
>   at junit.framework.TestResult$1.protect(TestResult.java:110)
>   at junit.framework.TestResult.runProtected(TestResult.java:128)
>   at junit.framework.TestResult.run(TestResult.java:113)
>   at junit.framework.TestCase.run(TestCase.java:124)
>   at junit.framework.TestSuite.runTest(TestSuite.java:243)
>   at junit.framework.TestSuite.run(TestSuite.java:238)
>   at 
> org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:83)
>   at org.junit.runners.Suite.runChild(Suite.java:128)
>   at org.junit.runners.Suite.runChild(Suite.java:24)
>   at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>   at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
>   at java.util.concurrent.FutureTask.run(FutureTask.java:166)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
>   at java.lang.Thread.run(Thread.java:722)
> testConfigDefaultPropsToAction
> org.apache.oozie.command.CommandException: E0803: IO error, Illegal character 
> in authority at index 7: 
> file://D:\w\oozie\sharelib\streaming\target\test-data\oozietests\org.apache.oozie.action.hadoop.TestMapReduceActionExecutor\testConfigDefaultPropsToAction\workflow.xml
>   at 
> org.apache.oozie.command.wf.SubmitXCommand.execute(SubmitXCommand.java:278)
>   at 
> org.apache.oozie.command.wf.SubmitXCommand.execute(SubmitXCommand.java:76)
>   at org.apache.oozie.command.XCommand.call(XCommand.java:286)
>   at 
> org.apache.oozie.action.hadoop.TestMapReduceActionExecutor.testConfigDefaultPropsToAction(TestMapReduceActionExecutor.java:139)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:601)
>   at junit.framework.TestCase.runTest(TestCase.java:168)
>   at junit.framework.TestCase.runBare(TestCase.java:134)
>   at junit.framework.TestResult$1.protect(TestResult.java:110)
>   at junit.framework.TestResult.runProtected(TestResult.java:128)
>   at junit.framework.TestResult.run(TestResult.java:113)
>   at junit.framework.TestCase.run(TestCase.java:124)
>   at 

[jira] [Commented] (OOZIE-2617) Read properties files in action configurations

[Taklon Stephen Wu (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15405138#comment-15405138
 ] 

Taklon Stephen Wu commented on OOZIE-2617:
--

testCoordMaterializeTriggerService3 is not related to my changes

> Read properties files in action configurations
> --
>
> Key: OOZIE-2617
> URL: https://issues.apache.org/jira/browse/OOZIE-2617
> Project: Oozie
>  Issue Type: Improvement
>  Components: core
>Affects Versions: trunk
>Reporter: Taklon Stephen Wu
>  Labels: patch
> Fix For: trunk
>
> Attachments: OOZIE-2617-1.patch, OOZIE-2617-2.patch, 
> OOZIE-2617-3.patch
>
>
> Current logic of acton configuration only READ xml files within Oozie action 
> directory, but for pig and other applications, properties files are supported 
> and may be default configuration file format. To simplify the logic of 
> reusing these properties files when using Oozie with them, we add properties 
> file reading in action conf.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2569) Adding yarn-site, core-site, hdfs-site and mapred-site into spark launcher

[Satish Subhashrao Saley (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15405063#comment-15405063
 ] 

Satish Subhashrao Saley commented on OOZIE-2569:


Can you make it optional in SparkMain like its in ShellMain?
{code}
if (actionConf.getBoolean(CONF_OOZIE_SHELL_SETUP_HADOOP_CONF_DIR, false)) 
{code}
You would like to change the name of constant to 
{{CONF_OOZIE_SPARK_SETUP_HADOOP_CONF_DIR="oozie.action.spark.setup.hadoop.conf.dir"}}

> Adding yarn-site, core-site, hdfs-site and mapred-site into spark launcher 
> ---
>
> Key: OOZIE-2569
> URL: https://issues.apache.org/jira/browse/OOZIE-2569
> Project: Oozie
>  Issue Type: Bug
>  Components: core
>Reporter: abhishek bafna
>Assignee: abhishek bafna
> Attachments: OOZIE-2569-00.patch
>
>
> Oozie Spark action requires yarn-site, core-site, hdfs-site and mapred-site 
> to be present in the classpath for launching the spark applications. 
> Currently we manually copy these files into oozie spark sharelib.
> Until it is fixed through the spark, we can copy the 
> {{oozie.action.conf.xml}} file as yarn-site, core-site, hdfs-site and 
> mapred-site, which will avoid manual steps of copying those files.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2617) Read properties files in action configurations

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15405054#comment-15405054
 ] 

Hadoop QA commented on OOZIE-2617:
--

Testing JIRA OOZIE-2617

Cleaning local git workspace



{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:red}-1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:red}-1{color} the patch contains 1 line(s) longer than 132 
characters
.{color:green}+1{color} the patch does adds/modifies 2 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT 
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc 
warnings
{color:red}-1 COMPILE{color}
.{color:red}-1{color} HEAD does not compile
.{color:red}-1{color} patch does not compile
.{color:green}+1{color} the patch does not seem to introduce new javac 
warnings
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA 
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:red}-1 TESTS{color}
.Tests run: 1795
.Tests failed: 1
.Tests errors: 0

.The patch failed the following testcases:

.  
testCoordMaterializeTriggerService3(org.apache.oozie.service.TestCoordMaterializeTriggerService)

{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch 


{color:red}*-1 Overall result, please check the reported -1(s)*{color}


The full output of the test-patch run is available at

.   https://builds.apache.org/job/oozie-trunk-precommit-build/3174/

> Read properties files in action configurations
> --
>
> Key: OOZIE-2617
> URL: https://issues.apache.org/jira/browse/OOZIE-2617
> Project: Oozie
>  Issue Type: Improvement
>  Components: core
>Affects Versions: trunk
>Reporter: Taklon Stephen Wu
>  Labels: patch
> Fix For: trunk
>
> Attachments: OOZIE-2617-1.patch, OOZIE-2617-2.patch, 
> OOZIE-2617-3.patch
>
>
> Current logic of acton configuration only READ xml files within Oozie action 
> directory, but for pig and other applications, properties files are supported 
> and may be default configuration file format. To simplify the logic of 
> reusing these properties files when using Oozie with them, we add properties 
> file reading in action conf.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Build failed in Jenkins: oozie-trunk-precommit-build #3174

[Apache Jenkins Server]

See 

Changes:

[rkanter] OOZIE-2555 Oozie SSL enable setup does not return port for admin

[rkanter] OOZIE-2172 ZooKeeper Security Tests failed with JVM IBM JAVA

[rkanter] OOZIE-2349 Method getCoordJobInfo(String jobId, String filter, int

[rkanter] OOZIE-2497 Some tests fail on windows due to hard coded URIs

[rkanter] OOZIE-2516 Update web service documentation for jobs API 
(abhishekbafna

--
[...truncated 8150 lines...]
[INFO] Tests are skipped.
[INFO] 
[INFO] --- maven-jar-plugin:2.3.1:jar (default-jar) @ oozie-tools ---
[INFO] Building jar: 

[INFO] 
[INFO] --- maven-site-plugin:2.0-beta-6:attach-descriptor (attach-descriptor) @ 
oozie-tools ---
[INFO] 
[INFO] --- maven-assembly-plugin:2.2.1:single (default-cli) @ oozie-tools ---
[INFO] Reading assembly descriptor: ../src/main/assemblies/tools.xml
[WARNING] The following patterns were never triggered in this artifact 
exclusion filter:
o  '*:*:pom:*'

[INFO] Copying files to 

[WARNING] Assembly file: 

 is not a regular file (it may be a directory). It cannot be attached to the 
project build for installation or deployment.
[INFO] 
[INFO] 
[INFO] Building Apache Oozie MiniOozie 4.3.0-SNAPSHOT
[INFO] 
[INFO] 
[INFO] --- maven-remote-resources-plugin:1.5:process (default) @ oozie-mini ---
[INFO] 
[INFO] --- maven-resources-plugin:2.5:resources (default-resources) @ 
oozie-mini ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:compile (default-compile) @ oozie-mini 
---
[INFO] No sources to compile
[INFO] 
[INFO] --- maven-resources-plugin:2.5:testResources (default-testResources) @ 
oozie-mini ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 3 resources
[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:testCompile (default-testCompile) @ 
oozie-mini ---
[INFO] Nothing to compile - all classes are up to date
[INFO] 
[INFO] --- maven-surefire-plugin:2.12.2:test (default-test) @ oozie-mini ---
[INFO] Tests are skipped.
[INFO] 
[INFO] --- maven-jar-plugin:2.3.1:jar (default-jar) @ oozie-mini ---
[INFO] Building jar: 

[INFO] 
[INFO] --- maven-site-plugin:2.0-beta-6:attach-descriptor (attach-descriptor) @ 
oozie-mini ---
[INFO] 
[INFO] --- maven-assembly-plugin:2.2.1:single (default-cli) @ oozie-mini ---
[INFO] Reading assembly descriptor: src/main/assemblies/empty.xml
[INFO] 
[INFO] 
[INFO] Building Apache Oozie Distro 4.3.0-SNAPSHOT
[INFO] 
[INFO] 
[INFO] --- maven-remote-resources-plugin:1.5:process (default) @ oozie-distro 
---
[INFO] 
[INFO] --- maven-resources-plugin:2.5:resources (default-resources) @ 
oozie-distro ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:compile (default-compile) @ oozie-distro 
---
[INFO] No sources to compile
[INFO] 
[INFO] --- maven-resources-plugin:2.5:testResources (default-testResources) @ 
oozie-distro ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:testCompile (default-testCompile) @ 
oozie-distro ---
[INFO] No sources to compile
[INFO] 
[INFO] --- maven-surefire-plugin:2.12.2:test (default-test) @ oozie-distro ---
[INFO] Tests are skipped.
[INFO] 
[INFO] --- maven-jar-plugin:2.3.1:jar (default-jar) @ oozie-distro ---
[INFO] Building jar: 

[jira] [Commented] (OOZIE-2496) Testcase failure due to environment specific line.separator value

[Robert Kanter (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15405015#comment-15405015
 ] 

Robert Kanter commented on OOZIE-2496:
--

This seems fine to me.  However, I'm concerned about the overhead of having to 
call {{System.getProperty("line.separator")}} so much; it's in a while loop.  
Perhaps we should cache the value in a static variable?  The tests don't really 
matter, so it's mainly just {{TimestampedMessageParser}}.

> Testcase failure due to environment specific line.separator value
> -
>
> Key: OOZIE-2496
> URL: https://issues.apache.org/jira/browse/OOZIE-2496
> Project: Oozie
>  Issue Type: Bug
>  Components: tests
> Environment: Windows
>Reporter: abhishek bafna
>Assignee: abhishek bafna
>  Labels: patch
> Attachments: OOZIE-2496-01.patch, OOZIE-2496-02.patch
>
>
> The "line.separator" returns operating system specific line separator. The 
> following testcase uses hard coded "\n" while comparing the return values in 
> the assertion, which makes them to fail on the windows environment.
> {noformat}
> org.apache.oozie.client.TestOozieCLI#testOozieStatus
> org.apache.oozie.client.TestOozieCLI#testServerBuildVersion
> org.apache.oozie.client.TestOozieCLI#testClientBuildVersion
> {noformat}
> In the following testcase, we are using "line.separator" where as 
> _TimestampedMessageParser_ sends the logs after appending "\n".
> {noformat}
> org.apache.oozie.util.TestXLogUserFilterParam#testloglevel_DEBUF_INFO
> org.apache.oozie.util.TestXLogUserFilterParam#testloglevel_Error
> org.apache.oozie.util.TestXLogUserFilterParam#testTextSearch
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: [SCOPE] Oozie 4.3.0 Release

[Robert Kanter]

Rohini, that sounds good to me.  Especially because we're on "master"
branch but "trunk" version in JIRA and it gets a little confusing.  Instead
of deleting "trunk", I think we're better off just renaming it to the
release number.  Otherwise, we'll have to re-label any current "trunk"
fields.  We can use 4.4.0 for now, but I imagine that the next release
might end up being 5.0.0 (with Oozie on Yarn); we can always rename it
later.

I'm not sure if there's a specific rule here, but I suppose we should
probably have a vote in the private list to make things official as this is
a release-management-related change.  Can you send an email?

Abhishek, thanks for putting this all together.  I think Rohini's
suggestion of updating the JIRAs is the right next step.  You're probably
best off using the bulk change feature instead of doing each JIRA
individually.  I should have time to do a bunch of reviews this week, I've
actually already started.  I should have some time to continue doing more
reviews.  As a general thing, we've been pretty slow about getting these
done :(

- Robert

On Tue, Aug 2, 2016 at 4:25 PM, Rohini Palaniswamy 
wrote:

> I also did mark some of the enhancement jiras that Satish was working on
> and which might take time to review to 4.4.0
>
> Regards,
> Rohini
>
> On Tue, Aug 2, 2016 at 3:01 PM, Rohini Palaniswamy <
> rohini.adi...@gmail.com>
> wrote:
>
> > Abhishek,
> > Can you mark all jiras already committed to trunk as 4.3.0 and those
> > planned in the scope document as 4.3.0? This will help filter on those
> > and review instead of referring to a google doc. I can pick up 10-15
> > jiras for review end of this week. We should try to get majority of
> patches
> > in the next 10 days and branch by Aug 12th. You can then use the 3rd week
> > for any pending critical issues and rolling the release candidate.
> >
> > Robert,
> >   I am thinking, that we should get rid of the trunk version and use
> > release numbers when committing from the beginning like all other
> projects
> > do. Unnecessary step to change from trunk to release version for every
> > release. If folks are in agreement, then will delete the trunk version
> > altogether. Does that require a mail to private?
> >
> > Regards,
> > Rohini
> >
> > On Fri, Jul 29, 2016 at 3:38 AM, Abhishek Bafna 
> > wrote:
> >
> >> Hi All,
> >>
> >> As we prepared a Oozie 4.3.0 scope document <
> >>
> https://docs.google.com/document/d/1E1vGTul0hrWC4R8EgCRkO3ZmGNT5GmFsuHQZmrT63Ow/edit#heading=h.w9ycxfry7sfu
> >
> >> and listed all the things which we wanted to include in the release. In
> the
> >> last few days, I spent some time and reviewed some of the patches and
> >> provided my feedback. I will continue my efforts to review more patches.
> >> These patches needs to be reviewed by committers and get committed. I
> >> request other contributors also, review some of the patches and provide
> >> their feedback.
> >>
> >> I am thinking, may be, we can decide upon some fixed (approximate)
> >> release date and start working accordingly. Because, If we wait for all
> the
> >> listed items to be reviewed and committed, then it might take a lot more
> >> time. I am thinking we should target third week of August to complete
> and
> >> release.
> >>
> >> Feel free to provide your feedback and suggestions.
> >>
> >> Thanks,
> >> Abhishek
> >> > On Jul 21, 2016, at 4:39 PM, Abhishek Bafna 
> >> wrote:
> >> >
> >> > Hi,
> >> >
> >> > I looked into the available patches and prepared the list. I have put,
> >> scope content which we have discussed till now, into a word document.
> (For
> >> the better formatting and easy to read and comment).
> >> >
> >> > I have noted the additional Jira into the list. Added two more Jira
> >> related to spark (Scala version and Spark 2.0 property).
> >> >
> >> > Please take a look at the document (link below) and provide your
> >> comments.
> >> >
> >> > If this looks good, we can start the release process.
> >> >
> >> > Scope Document:
> >>
> https://docs.google.com/document/d/1E1vGTul0hrWC4R8EgCRkO3ZmGNT5GmFsuHQZmrT63Ow/edit#
> >> >
> >> > Thanks,
> >> > Abhishek
> >> >
> >> >> On Jul 21, 2016, at 5:07 AM, Robert Kanter 
> >> wrote:
> >> >>
> >> >> That list looks good to me.  The only one I'm not sure of is
> OOZIE-2306
> >> >> (Oozie Health Check).  It's a new feature and it might make sense to
> >> let it
> >> >> "bake" a while first?  I'm also not sure it's quite ready yet.
> >> >>
> >> >> I'd also like to add a few others to the list:
> >> >> - OOZIE-2273 (MiniOozie does not work outside of Oozie)
> >> >> - The subtasks under OOZIE-2036 (Drop support for Java 1.6)
> >> >> --- OOZIE-2037 (Add TLSv1.1,TLSv1.2)
> >> >> --- OOZIE-1793 (Improve find bugs reporting for Oozie)
> >> >> --- OOZIE-2552 (Update ActiveMQ version for security and other fixes)
> >> >>
> >> >> There's a number of other Patch Available JIRAs not on 

[jira] [Updated] (OOZIE-2259) Create a callback action

[Rohini Palaniswamy (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-2259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rohini Palaniswamy updated OOZIE-2259:
--
Fix Version/s: 4.3.0

> Create a callback action 
> -
>
> Key: OOZIE-2259
> URL: https://issues.apache.org/jira/browse/OOZIE-2259
> Project: Oozie
>  Issue Type: New Feature
>  Components: action
>Reporter: Jaydeep Vishwakarma
>Assignee: Jaydeep Vishwakarma
> Fix For: 4.3.0
>
> Attachments: OOZIE-2259-v1.patch, OOZIE-2259-v3.patch, 
> OOZIE-2259-v4.patch, OOZIE-2259-v5.patch, OOZIE-2259-v8.patch, 
> OOZIE-2259-v9.patch, OOZIE-2259_v6.patch, OOZIE-2259_v7.patch
>
>
> Need an action to send notification to external server by oozie. We should be 
> able to do multiple types of callback, Currently I know jms and http call. It 
> should suppose to have capability to call diffrent types of methods along 
> with n number of arguments. 
> The sample workflow with callback action 
> {code:xml}
> 
> ...
> 
> 
>   [HOST]
>   [METHOD]
>   
>   [KEY][VALUE]
>   
> ...
> 
> ...
> 
> ...
> 
> {code}
> HOST : by the host system can figure out if it is http or jms callback 
> action. System will send the notification to that host.
> METHOD : it can be POST/GET/QUEUE/TOPIC



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: [SCOPE] Oozie 4.3.0 Release

[Rohini Palaniswamy]

I also did mark some of the enhancement jiras that Satish was working on
and which might take time to review to 4.4.0

Regards,
Rohini

On Tue, Aug 2, 2016 at 3:01 PM, Rohini Palaniswamy 
wrote:

> Abhishek,
> Can you mark all jiras already committed to trunk as 4.3.0 and those
> planned in the scope document as 4.3.0? This will help filter on those
> and review instead of referring to a google doc. I can pick up 10-15
> jiras for review end of this week. We should try to get majority of patches
> in the next 10 days and branch by Aug 12th. You can then use the 3rd week
> for any pending critical issues and rolling the release candidate.
>
> Robert,
>   I am thinking, that we should get rid of the trunk version and use
> release numbers when committing from the beginning like all other projects
> do. Unnecessary step to change from trunk to release version for every
> release. If folks are in agreement, then will delete the trunk version
> altogether. Does that require a mail to private?
>
> Regards,
> Rohini
>
> On Fri, Jul 29, 2016 at 3:38 AM, Abhishek Bafna 
> wrote:
>
>> Hi All,
>>
>> As we prepared a Oozie 4.3.0 scope document <
>> https://docs.google.com/document/d/1E1vGTul0hrWC4R8EgCRkO3ZmGNT5GmFsuHQZmrT63Ow/edit#heading=h.w9ycxfry7sfu>
>> and listed all the things which we wanted to include in the release. In the
>> last few days, I spent some time and reviewed some of the patches and
>> provided my feedback. I will continue my efforts to review more patches.
>> These patches needs to be reviewed by committers and get committed. I
>> request other contributors also, review some of the patches and provide
>> their feedback.
>>
>> I am thinking, may be, we can decide upon some fixed (approximate)
>> release date and start working accordingly. Because, If we wait for all the
>> listed items to be reviewed and committed, then it might take a lot more
>> time. I am thinking we should target third week of August to complete and
>> release.
>>
>> Feel free to provide your feedback and suggestions.
>>
>> Thanks,
>> Abhishek
>> > On Jul 21, 2016, at 4:39 PM, Abhishek Bafna 
>> wrote:
>> >
>> > Hi,
>> >
>> > I looked into the available patches and prepared the list. I have put,
>> scope content which we have discussed till now, into a word document. (For
>> the better formatting and easy to read and comment).
>> >
>> > I have noted the additional Jira into the list. Added two more Jira
>> related to spark (Scala version and Spark 2.0 property).
>> >
>> > Please take a look at the document (link below) and provide your
>> comments.
>> >
>> > If this looks good, we can start the release process.
>> >
>> > Scope Document:
>> https://docs.google.com/document/d/1E1vGTul0hrWC4R8EgCRkO3ZmGNT5GmFsuHQZmrT63Ow/edit#
>> >
>> > Thanks,
>> > Abhishek
>> >
>> >> On Jul 21, 2016, at 5:07 AM, Robert Kanter 
>> wrote:
>> >>
>> >> That list looks good to me.  The only one I'm not sure of is OOZIE-2306
>> >> (Oozie Health Check).  It's a new feature and it might make sense to
>> let it
>> >> "bake" a while first?  I'm also not sure it's quite ready yet.
>> >>
>> >> I'd also like to add a few others to the list:
>> >> - OOZIE-2273 (MiniOozie does not work outside of Oozie)
>> >> - The subtasks under OOZIE-2036 (Drop support for Java 1.6)
>> >> --- OOZIE-2037 (Add TLSv1.1,TLSv1.2)
>> >> --- OOZIE-1793 (Improve find bugs reporting for Oozie)
>> >> --- OOZIE-2552 (Update ActiveMQ version for security and other fixes)
>> >>
>> >> There's a number of other Patch Available JIRAs not on your list
>> (including
>> >> the ones I mentioned above).  While we don't need to include everything
>> >> here (and we probably shouldn't nor do we have the review bandwidth),
>> did
>> >> you consider each of them when creating the list?  I currently see 97
>> >> JIRAs: https://s.apache.org/oozie-patch-available
>> >> I think we should include all Spark Action fixes/improvements, given
>> how
>> >> popular it seems to be.
>> >>
>> >> thanks
>> >> - Robert
>> >>
>> >>
>> >>
>> >>
>> >> On Wed, Jul 20, 2016 at 5:00 AM, Abhishek Bafna 
>> >> wrote:
>> >>
>> >>> Hi All,
>> >>>
>> >>> Below is the list of issues, picked from currently available patches.
>> I
>> >>> could not decide up any 'open' issue, If there is anything, please
>> reply
>> >>> here.
>> >>>
>> >>> We can 'drop/add' more if required based on the feedback. Important
>> is,
>> >>> how many of these we will be able to review and commit.
>> >>>
>> >>> I will starting looking into these issues and provide feedback.
>> >>>
>> ---
>> >>> OOZIE-1978  Forkjoin validation code is ridiculously slow in some
>> cases
>> >>> OOZIE-2508  Documentation change for Coord action rerun
>> [OOZIE-1735]
>> >>> OOZIE-2583  oozie throws EL Exception when reference variable name
>> >>> containing dot
>> >>> 

[jira] [Updated] (OOZIE-2461) Workflow, Coordinator and Bundle job querying should have last modified filter

[Rohini Palaniswamy (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-2461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rohini Palaniswamy updated OOZIE-2461:
--
Fix Version/s: 4.4.0

> Workflow, Coordinator and Bundle job querying should have last modified filter
> --
>
> Key: OOZIE-2461
> URL: https://issues.apache.org/jira/browse/OOZIE-2461
> Project: Oozie
>  Issue Type: Bug
>Reporter: Satish Subhashrao Saley
>Assignee: Satish Subhashrao Saley
> Fix For: 4.4.0
>
> Attachments: OOZIE-2461-1.patch, OOZIE-2461-10.patch, 
> OOZIE-2461-11.patch, OOZIE-2461-6.patch, OOZIE-2461-7.patch, 
> OOZIE-2461-8.patch, OOZIE-2461-9.patch
>
>
> To get currently running coordinator and id, one user had to do
> http://localhost:11000/oozie/v1/jobs?jobtype=coord=user%3satish_1.0%3B=1=3000
> They could not use name in the filter as they include a version and keep 
> changing. For eg:
> urs_satish_filter-0.1-daily-coord
> urs_puru_service-0.4-hourly-coord
> It would be good to have last modified filter to get recently active 
> coordinators.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OOZIE-2457) Oozie log parsing regex consume more than 90% cpu

[Rohini Palaniswamy (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-2457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rohini Palaniswamy updated OOZIE-2457:
--
Fix Version/s: 4.4.0

> Oozie log parsing regex consume more than 90% cpu
> -
>
> Key: OOZIE-2457
> URL: https://issues.apache.org/jira/browse/OOZIE-2457
> Project: Oozie
>  Issue Type: Bug
>Reporter: Satish Subhashrao Saley
>Assignee: Satish Subhashrao Saley
>Priority: Minor
> Fix For: 4.4.0
>
> Attachments: OOZIE-2457-1.patch, OOZIE-2457-2.patch, 
> OOZIE-2457-3.patch
>
>
> http-0.0.0.0-4080-26  TID=62215  STATE=RUNNABLE  CPU_TIME=1992 (92.59%)  
> USER_TIME=1990 (92.46%) Allocted: 269156584
> java.util.regex.Pattern$Curly.match0(Pattern.java:4170)
> java.util.regex.Pattern$Curly.match(Pattern.java:4132)
> java.util.regex.Pattern$GroupHead.match(Pattern.java:4556)
> java.util.regex.Matcher.match(Matcher.java:1221)
> java.util.regex.Matcher.matches(Matcher.java:559)
> org.apache.oozie.util.XLogFilter.matches(XLogFilter.java:136)
> 
> org.apache.oozie.util.TimestampedMessageParser.parseNextLine(TimestampedMessageParser.java:145)
> 
> org.apache.oozie.util.TimestampedMessageParser.increment(TimestampedMessageParser.java:92)
> Regex 
> {code}
> (.* USER\[[^\]]*\] GROUP\[[^\]]*\] TOKEN\[[^\]]*\] APP\[[^\]]*\] 
> JOB\[000-150625114739728-oozie-puru-W\] ACTION\[[^\]]*\] .*)
> {code}
> For single line parsing we use two regex.
> 1. 
> {code}
> public ArrayList splitLogMessage(String logLine) {
> Matcher splitter = SPLITTER_PATTERN.matcher(logLine);
> if (splitter.matches()) {
> ArrayList logParts = new ArrayList();
> logParts.add(splitter.group(1));// timestamp
> logParts.add(splitter.group(2));// log level
> logParts.add(splitter.group(3));// Log Message
> return logParts;
> }
> else {
> return null;
> }
> }
> {code}
> 2.
> {code}
>  public boolean matches(ArrayList logParts) {
> if (getStartDate() != null) {
> if (logParts.get(0).substring(0, 
> 19).compareTo(getFormattedStartDate()) < 0) {
> return false;
> }
> }
> String logLevel = logParts.get(1);
> String logMessage = logParts.get(2);
> if (this.logLevels == null || 
> this.logLevels.containsKey(logLevel.toUpperCase())) {
> Matcher logMatcher = filterPattern.matcher(logMessage);
> return logMatcher.matches();
> }
> else {
> return false;
> }
> }
> {code}
> Also there is repetitive parsing  for same log message in
> {code}
> private String parseTimestamp(String line) {
> String timestamp = null;
> ArrayList logParts = filter.splitLogMessage(line);
> if (logParts != null) {
> timestamp = logParts.get(0);
> }
> return timestamp;
> }
> {code}
> where the {{line}} has already parsed using regex and we already know the 
> {{logParts}} if any.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OOZIE-2473) Connection pool for SMTP connection

[Rohini Palaniswamy (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-2473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rohini Palaniswamy updated OOZIE-2473:
--
Fix Version/s: 4.4.0

> Connection pool for SMTP connection
> ---
>
> Key: OOZIE-2473
> URL: https://issues.apache.org/jira/browse/OOZIE-2473
> Project: Oozie
>  Issue Type: Bug
>Reporter: Satish Subhashrao Saley
>Assignee: Satish Subhashrao Saley
> Fix For: 4.4.0
>
> Attachments: OOZIE-2473-1.patch, OOZIE-2473-1.patch, 
> OOZIE-2473-2.patch
>
>
> Currently, to send an email we setup new connection every time which seems 
> costly. It would be good to have a connection pool.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OOZIE-2501) ZK reentrant lock doesn't work for few cases

[Rohini Palaniswamy (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-2501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rohini Palaniswamy updated OOZIE-2501:
--
Fix Version/s: 4.3.0

> ZK reentrant lock doesn't work for few cases
> 
>
> Key: OOZIE-2501
> URL: https://issues.apache.org/jira/browse/OOZIE-2501
> Project: Oozie
>  Issue Type: Bug
>Reporter: Purshotam Shah
>Assignee: Purshotam Shah
> Fix For: 4.3.0
>
> Attachments: OOZIE-2501-V2.patch
>
>
> We will have an issue when oozie trying to acquire a lock and at the same 
> time, some other thread is releasing the same lock .
> acquireLock will wait for 5 sec to acquire the lock. It will bypass the 
> synchronized block and get lockEntry from the hashmap.
> While it waiting for 5 sec to acquire the lock, other thread releases the 
> lock and may execute the release code which will remove  lockEntry from the 
> map.
> If some other command from same thread tries to acquire the lock, it will 
> create a new InterProcessReadWriteLock object and use that for acquiring the 
> lock. 
> Logic for lock acquiring.
> {code}
>  public LockToken getWriteLock(String resource, long wait) throws 
> InterruptedException {
> InterProcessReadWriteLock lockEntry;
> synchronized (zkLocks) {
> if (zkLocks.containsKey(resource)) {
> lockEntry = zkLocks.get(resource);
> }
> else {
> lockEntry = new InterProcessReadWriteLock(zk.getClient(), 
> LOCKS_NODE + "/" + resource);
> zkLocks.put(resource, lockEntry);
> }
> }
> InterProcessMutex writeLock = lockEntry.writeLock();
> return acquireLock(wait, writeLock, resource);
> }
> {code}
> Logic for lock releasing
> {code}
> public void release() {
> try {
> lock.release();
> if (zkLocks.get(resource) == null) {
> return;
> }
> if (!isLockHeld()) {
> synchronized (zkLocks) {
> if (zkLocks.get(resource) != null) {
> if (!isLockHeld()) {
> zkLocks.remove(resource);
> }
> }
> }
> }
> }
> catch (Exception ex) {
> LOG.warn("Could not release lock: " + ex.getMessage(), ex);
> }
> }
> {code}
> Curator code to acquire lock.
> {code}
> private boolean internalLock(long time, TimeUnit unit) throws Exception
> {
> /*
>Note on concurrency: a given lockData instance
>can be only acted on by a single thread so locking isn't necessary
> */
> Thread  currentThread = Thread.currentThread();
> LockDatalockData = threadData.get(currentThread);
> if ( lockData != null )
> {
> // re-entering
> lockData.lockCount.incrementAndGet();
> return true;
> }
> String lockPath = internals.attemptLock(time, unit, 
> getLockNodeBytes());
> if ( lockPath != null )
> {
> LockDatanewLockData = new LockData(currentThread, 
> lockPath);
> threadData.put(currentThread, newLockData);
> return true;
> }
> return false;
> }
> {code}
> The approach we have followed is to use map with weakvalue. Once the lock is 
> unreachable. GC will remove it from the map. We don't have to explicitly 
> remove it.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2589) CompletedActionXCommand priority param is of no effect

[Robert Kanter (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15404991#comment-15404991
 ] 

Robert Kanter commented on OOZIE-2589:
--

We should think about this a bit more.  The 4-argument constructor is called by 
{{DagEngine}}, and uses priority 2, but is currently overridden (as this JIRA 
points out) to priority 1.  The 3-argument constructor is hardcoded to priority 
1.  We should figure out which priority this should normally be called at.  



> CompletedActionXCommand priority param is of no effect
> --
>
> Key: OOZIE-2589
> URL: https://issues.apache.org/jira/browse/OOZIE-2589
> Project: Oozie
>  Issue Type: Bug
>  Components: core
>Affects Versions: trunk
>Reporter: Linflytang
>  Labels: patch
> Fix For: trunk
>
> Attachments: 0001-Fixed-CompletedActionXCommand-priority-param.patch
>
>
> CompletedActionXCommand priority param is hard coding with 1, but create the 
> action with 2 (HIGH_PRIORITY).
> patch:https://github.com/apache/oozie/pull/20/files



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OOZIE-2623) Oozie should use a dummy OutputFormat

[Rohini Palaniswamy (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-2623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rohini Palaniswamy updated OOZIE-2623:
--
Description: 
After fixing OOZIE-2475, killed actions have this exception.

{code}
2016-06-13 20:49:46,068 [uber-SubtaskRunner] WARN  
org.apache.hadoop.mapred.LocalContainerLauncher  - Exception running local 
(uberized) 'child' : 
org.apache.hadoop.ipc.RemoteException(java.io.FileNotFoundException): Invalid 
inode path: 
/user/rohinip/oozie_LL/0266574-160613173929807-oozie_LL-W/pig1--pig/output/_temporary/1/_temporary/attempt_1464982500298_46637_m_00_0/part-0
 (id=316134636)
{code}

This is because it currently uses FileOuputFormat and sets output directory to 
/output which is unnecessary.  This should save some unnecessary 
file creation and move operations.

  was:
After fixing OOZIE-2475, killed actions have this exception.

{code}
2016-06-13 20:49:46,068 [uber-SubtaskRunner] WARN  
org.apache.hadoop.mapred.LocalContainerLauncher  - Exception running local 
(uberized) 'child' : 
org.apache.hadoop.ipc.RemoteException(java.io.FileNotFoundException): Invalid 
inode path: 
/user/rohinip/oozie_LL/0266574-160613173929807-oozie_LL-W/pig1--pig/output/_temporary/1/_temporary/attempt_1464982500298_46637_m_00_0/part-0
 (id=316134636)
{code}

This is because it currently uses FileOuputFormat and sets output directory to 
/output which is unnecessary. 


> Oozie should use a dummy OutputFormat
> -
>
> Key: OOZIE-2623
> URL: https://issues.apache.org/jira/browse/OOZIE-2623
> Project: Oozie
>  Issue Type: Bug
>Reporter: Satish Subhashrao Saley
>Assignee: Satish Subhashrao Saley
> Attachments: OOZIE-2623-1.patch, OOZIE-2623-2.patch, 
> OOZIE-2623-3.patch
>
>
> After fixing OOZIE-2475, killed actions have this exception.
> {code}
> 2016-06-13 20:49:46,068 [uber-SubtaskRunner] WARN  
> org.apache.hadoop.mapred.LocalContainerLauncher  - Exception running local 
> (uberized) 'child' : 
> org.apache.hadoop.ipc.RemoteException(java.io.FileNotFoundException): Invalid 
> inode path: 
> /user/rohinip/oozie_LL/0266574-160613173929807-oozie_LL-W/pig1--pig/output/_temporary/1/_temporary/attempt_1464982500298_46637_m_00_0/part-0
>  (id=316134636)
> {code}
> This is because it currently uses FileOuputFormat and sets output directory 
> to /output which is unnecessary.  This should save some 
> unnecessary file creation and move operations.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2081) WorkflowJob notification to include coordinator action id

[Robert Kanter (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15404947#comment-15404947
 ] 

Robert Kanter commented on OOZIE-2081:
--

Looks good overall.  Just one minor thing:
- The docs should explain the behavior when there is no parent id (i.e. it's 
replaced with an empty string).  Otherwise, user's processing the URL might not 
be aware that they need to handle that case.

> WorkflowJob notification to include coordinator action id 
> --
>
> Key: OOZIE-2081
> URL: https://issues.apache.org/jira/browse/OOZIE-2081
> Project: Oozie
>  Issue Type: Improvement
>  Components: core
>Reporter: Venkatesan Ramachandran
>Assignee: Azrael Seoeun
> Fix For: trunk, 4.1.0, 4.0.1
>
> Attachments: OOZIE-2081.1.patch, OOZIE-2081.2.patch
>
>
> WorkflowJob notification to include parent_id (coordinator action id) if 
> present. This will help identify if a given workflow job is started by a 
> coordinator action.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2518) Oozie is creating many files and directories in /tmp per day

[Robert Kanter (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15404938#comment-15404938
 ] 

Robert Kanter commented on OOZIE-2518:
--

The only way to run into this problem is if you rebuild Oozie and/or the 
database very frequently.  Normal users shouldn't run into this, right?  The 
files should eventually be cleaned up by the OS.

> Oozie is creating many files and directories in /tmp per day
> 
>
> Key: OOZIE-2518
> URL: https://issues.apache.org/jira/browse/OOZIE-2518
> Project: Oozie
>  Issue Type: Improvement
>Affects Versions: 4.2.0
>Reporter: Sergey Svinarchuk
>Assignee: Sergey Svinarchuk
>  Labels: patch
> Fix For: trunk
>
> Attachments: OOZIE-2518.patch
>
>
> Oozie generate many tmp files, I have over 2000 files in /tmp from oozie, 
> that have been created over the last 7 days: 
> {code}
> drwxr-xr-x 3 root root   22 Apr 22 14:19 /tmp/oozie-war-packing-24575
> drwxr-xr-x 3 root root   22 Apr 22 14:15 /tmp/oozie-war-packing-12211
> drwxr-xr-x 3 root root   22 Apr 22 14:10 /tmp/oozie-war-packing-32238
> drwxr-xr-x 2 test test6 Apr 22 13:54 
> /tmp/oozie-test8457897583638353023.dir
> -rw-r--r-- 1 test test0 Apr 22 13:54 /tmp/ooziedb-2680155694357966339.sql
> drwxr-xr-x 2 test test6 Apr 22 13:53 /tmp/oozie-test142692242798969000.dir
> -rw-r--r-- 1 test test0 Apr 22 13:53 /tmp/ooziedb-9104680045020610294.sql
> drwxr-xr-x 2 test test6 Apr 22 13:53 
> /tmp/oozie-test5885601734493698256.dir
> -rw-r--r-- 1 test test0 Apr 22 13:53 /tmp/ooziedb-4675739133078261538.sql
> drwxr-xr-x 2 test test6 Apr 22 13:53 
> /tmp/oozie-test5565479595579419225.dir
> [root@test-node ext-conf]# ls -ldt /tmp/ooz* | tail
> drwxr-xr-x 3 root root   22 Apr 13 08:17 /tmp/oozie-war-packing-22705
> drwxr-xr-x 3 root root   22 Apr 13 08:16 /tmp/oozie-war-packing-18292
> drwxr-xr-x 3 root root   22 Apr 13 08:09 /tmp/oozie-war-packing-20289
> drwxr-xr-x 3 root root   22 Apr 13 08:09 /tmp/oozie-war-packing-18417
> drwxr-xr-x 3 root root   22 Apr 13 08:08 /tmp/oozie-war-packing-14712
> drwxr-xr-x 3 root root   22 Apr 13 08:05 /tmp/oozie-war-packing-25206
> drwxr-xr-x 3 root root   22 Apr 13 07:53 /tmp/oozie-war-packing-968
> drwxr-xr-x 3 root root   22 Apr 13 07:52 /tmp/oozie-war-packing-27771
> drwxr-xr-x 3 root root   22 Apr 13 07:45 /tmp/oozie-war-packing-30342
> drwxr-xr-x 3 root root   22 Apr 13 07:45 /tmp/oozie-war-packing-29086
> {code}
> This files can be moved into oozie tmp dir (something like this 
> /tmp/oozieTmp) and also we could cleanup old files



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2493) TestDiscp leaves action.xml behind on local filesystem

[Robert Kanter (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15404933#comment-15404933
 ] 

Robert Kanter commented on OOZIE-2493:
--

[~abhishekbafna], it looks like the action.xml file is still being left behind 
even with the patch.  Though the patch looks like it should work.  Can you try 
to look into it?

> TestDiscp leaves action.xml behind on local filesystem
> --
>
> Key: OOZIE-2493
> URL: https://issues.apache.org/jira/browse/OOZIE-2493
> Project: Oozie
>  Issue Type: Bug
>  Components: tests
>Reporter: abhishek bafna
>Assignee: abhishek bafna
>  Labels: patch
> Fix For: 4.3.0
>
> Attachments: OOZIE-2493-02.patch, OOZIE-2493.patch
>
>
> In the TestDistcpMain#testMain test case, it is deleting a file using HDFS 
> FileSystem instance which actually created on a local FileSystem.
> {code}
> // getting file hadoop system instance 
> FileSystem fs = getFileSystem();
> ...
> // creating a storing file on local file system
> File actionXml = new File(getTestCaseDir(), "action.xml");
> ...
> // Now trying to delete the file on hadoop file system
> fs.delete(new Path(getTestCaseDir(), "action.xml"), true);
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OOZIE-2493) TestDiscp leaves action.xml behind on local filesystem

[Robert Kanter (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-2493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robert Kanter updated OOZIE-2493:
-
Summary: TestDiscp leaves action.xml behind on local filesystem  (was: 
Remove action.xml file from local file system instead of hadoop file system)

> TestDiscp leaves action.xml behind on local filesystem
> --
>
> Key: OOZIE-2493
> URL: https://issues.apache.org/jira/browse/OOZIE-2493
> Project: Oozie
>  Issue Type: Bug
>  Components: tests
>Reporter: abhishek bafna
>Assignee: abhishek bafna
>  Labels: patch
> Fix For: 4.3.0
>
> Attachments: OOZIE-2493-02.patch, OOZIE-2493.patch
>
>
> In the TestDistcpMain#testMain test case, it is deleting a file using HDFS 
> FileSystem instance which actually created on a local FileSystem.
> {code}
> // getting file hadoop system instance 
> FileSystem fs = getFileSystem();
> ...
> // creating a storing file on local file system
> File actionXml = new File(getTestCaseDir(), "action.xml");
> ...
> // Now trying to delete the file on hadoop file system
> fs.delete(new Path(getTestCaseDir(), "action.xml"), true);
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2574) Oozie to support replication-enabled mysql urls

[Robert Kanter (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15404906#comment-15404906
 ] 

Robert Kanter commented on OOZIE-2574:
--

The patch seems okay to me, though it's funny that simply quoting the url is 
enough to fix it; I'd have thought OpenJPA or the driver could take care of 
that sort of thing :/

Anyway, were you able to verify that this works?  I don't have a 
replication-enabled MySQL handy.

> Oozie to support replication-enabled mysql urls
> ---
>
> Key: OOZIE-2574
> URL: https://issues.apache.org/jira/browse/OOZIE-2574
> Project: Oozie
>  Issue Type: Improvement
>  Components: core
>Reporter: abhishek bafna
>Assignee: abhishek bafna
> Attachments: OOZIE-2574-00.patch
>
>
> Currently Oozie does not support replication-enabled mysql urls. As per the 
> mysql JDBC specification we can use 
> {{jdbc:mysql:replication://master_host:port,slave_host:port/database}}.
> https://dev.mysql.com/doc/connector-j/5.1/en/connector-j-master-slave-replication-connection.html



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: [SCOPE] Oozie 4.3.0 Release

[Rohini Palaniswamy]

Abhishek,
Can you mark all jiras already committed to trunk as 4.3.0 and those
planned in the scope document as 4.3.0? This will help filter on those and
review instead of referring to a google doc. I can pick up 10-15 jiras for
review end of this week. We should try to get majority of patches in the
next 10 days and branch by Aug 12th. You can then use the 3rd week for any
pending critical issues and rolling the release candidate.

Robert,
  I am thinking, that we should get rid of the trunk version and use
release numbers when committing from the beginning like all other projects
do. Unnecessary step to change from trunk to release version for every
release. If folks are in agreement, then will delete the trunk version
altogether. Does that require a mail to private?

Regards,
Rohini

On Fri, Jul 29, 2016 at 3:38 AM, Abhishek Bafna 
wrote:

> Hi All,
>
> As we prepared a Oozie 4.3.0 scope document <
> https://docs.google.com/document/d/1E1vGTul0hrWC4R8EgCRkO3ZmGNT5GmFsuHQZmrT63Ow/edit#heading=h.w9ycxfry7sfu>
> and listed all the things which we wanted to include in the release. In the
> last few days, I spent some time and reviewed some of the patches and
> provided my feedback. I will continue my efforts to review more patches.
> These patches needs to be reviewed by committers and get committed. I
> request other contributors also, review some of the patches and provide
> their feedback.
>
> I am thinking, may be, we can decide upon some fixed (approximate) release
> date and start working accordingly. Because, If we wait for all the listed
> items to be reviewed and committed, then it might take a lot more time. I
> am thinking we should target third week of August to complete and release.
>
> Feel free to provide your feedback and suggestions.
>
> Thanks,
> Abhishek
> > On Jul 21, 2016, at 4:39 PM, Abhishek Bafna 
> wrote:
> >
> > Hi,
> >
> > I looked into the available patches and prepared the list. I have put,
> scope content which we have discussed till now, into a word document. (For
> the better formatting and easy to read and comment).
> >
> > I have noted the additional Jira into the list. Added two more Jira
> related to spark (Scala version and Spark 2.0 property).
> >
> > Please take a look at the document (link below) and provide your
> comments.
> >
> > If this looks good, we can start the release process.
> >
> > Scope Document:
> https://docs.google.com/document/d/1E1vGTul0hrWC4R8EgCRkO3ZmGNT5GmFsuHQZmrT63Ow/edit#
> >
> > Thanks,
> > Abhishek
> >
> >> On Jul 21, 2016, at 5:07 AM, Robert Kanter 
> wrote:
> >>
> >> That list looks good to me.  The only one I'm not sure of is OOZIE-2306
> >> (Oozie Health Check).  It's a new feature and it might make sense to
> let it
> >> "bake" a while first?  I'm also not sure it's quite ready yet.
> >>
> >> I'd also like to add a few others to the list:
> >> - OOZIE-2273 (MiniOozie does not work outside of Oozie)
> >> - The subtasks under OOZIE-2036 (Drop support for Java 1.6)
> >> --- OOZIE-2037 (Add TLSv1.1,TLSv1.2)
> >> --- OOZIE-1793 (Improve find bugs reporting for Oozie)
> >> --- OOZIE-2552 (Update ActiveMQ version for security and other fixes)
> >>
> >> There's a number of other Patch Available JIRAs not on your list
> (including
> >> the ones I mentioned above).  While we don't need to include everything
> >> here (and we probably shouldn't nor do we have the review bandwidth),
> did
> >> you consider each of them when creating the list?  I currently see 97
> >> JIRAs: https://s.apache.org/oozie-patch-available
> >> I think we should include all Spark Action fixes/improvements, given how
> >> popular it seems to be.
> >>
> >> thanks
> >> - Robert
> >>
> >>
> >>
> >>
> >> On Wed, Jul 20, 2016 at 5:00 AM, Abhishek Bafna 
> >> wrote:
> >>
> >>> Hi All,
> >>>
> >>> Below is the list of issues, picked from currently available patches. I
> >>> could not decide up any 'open' issue, If there is anything, please
> reply
> >>> here.
> >>>
> >>> We can 'drop/add' more if required based on the feedback. Important is,
> >>> how many of these we will be able to review and commit.
> >>>
> >>> I will starting looking into these issues and provide feedback.
> >>>
> ---
> >>> OOZIE-1978  Forkjoin validation code is ridiculously slow in some
> cases
> >>> OOZIE-2508  Documentation change for Coord action rerun
> [OOZIE-1735]
> >>> OOZIE-2583  oozie throws EL Exception when reference variable name
> >>> containing dot
> >>> OOZIE-2582  Populating external child Ids for action failures
> >>> OOZIE-2517  Add support for startCreatedTime and endCreatedTime
> >>> filters for coord and bundles
> >>> OOZIE-2512  ShareLibservice returns incorrect path for jar
> >>> OOZIE-2572  SLA DURATION miss not shown when job is running for
> longer
> >>> than expected time
> >>> OOZIE-2457 

[jira] [Updated] (OOZIE-2617) Read properties files in action configurations

[Taklon Stephen Wu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-2617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Taklon Stephen Wu updated OOZIE-2617:
-
Attachment: OOZIE-2617-3.patch

updated patch and remove unnecessary changes in pom.xml

> Read properties files in action configurations
> --
>
> Key: OOZIE-2617
> URL: https://issues.apache.org/jira/browse/OOZIE-2617
> Project: Oozie
>  Issue Type: Improvement
>  Components: core
>Affects Versions: trunk
>Reporter: Taklon Stephen Wu
>  Labels: patch
> Fix For: trunk
>
> Attachments: OOZIE-2617-1.patch, OOZIE-2617-2.patch, 
> OOZIE-2617-3.patch
>
>
> Current logic of acton configuration only READ xml files within Oozie action 
> directory, but for pig and other applications, properties files are supported 
> and may be default configuration file format. To simplify the logic of 
> reusing these properties files when using Oozie with them, we add properties 
> file reading in action conf.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 50424: Read properties files in action configurations

[TAKLON STEPHEN WU]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50424/
---

(Updated Aug. 2, 2016, 9:37 p.m.)


Review request for oozie.


Changes
---

updated twiki and comments inline


Bugs: OOZIE-2617
https://issues.apache.org/jira/browse/OOZIE-2617


Repository: oozie-git


Description
---

Current logic of acton configuration only READ xml files within Oozie action 
directory, but for pig and other applications, properties files are supported 
and may be default configuration file format. To simplify the logic of reusing 
these properties files when using Oozie with them, we add properties file 
reading in action conf.


Diffs (updated)
-

  core/src/main/java/org/apache/oozie/service/ActionConfFileType.java 
PRE-CREATION 
  core/src/main/java/org/apache/oozie/service/HadoopAccessorService.java 
f171ea3 
  core/src/test/java/org/apache/oozie/service/TestActionConfFileType.java 
PRE-CREATION 
  core/src/test/java/org/apache/oozie/service/TestHadoopAccessorService.java 
bbe2594 
  docs/src/site/twiki/AG_ActionConfiguration.twiki f8e6edc 

Diff: https://reviews.apache.org/r/50424/diff/


Testing
---

$ cd oozie/core
$ mvn clean install -Dtest=TestHadoopAccessorService 
-Dtest=TestActionConfFileType

Running org.apache.oozie.service.TestActionConfFileType
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 16.493 sec
Running org.apache.oozie.service.TestHadoopAccessorService
Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.003 sec

Results :

Tests run: 10, Failures: 0, Errors: 0, Skipped: 0

[INFO] 
[INFO] BUILD SUCCESS
[INFO] 
[INFO] Total time: 56.006 s
[INFO] Finished at: 2016-07-22T19:26:28-07:00
[INFO] Final Memory: 81M/1473M
[INFO] 


Thanks,

TAKLON STEPHEN WU

[jira] [Updated] (OOZIE-2617) Read properties files in action configurations

[Taklon Stephen Wu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-2617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Taklon Stephen Wu updated OOZIE-2617:
-
Attachment: OOZIE-2617-2.patch

updated twiki file and a comment inline

> Read properties files in action configurations
> --
>
> Key: OOZIE-2617
> URL: https://issues.apache.org/jira/browse/OOZIE-2617
> Project: Oozie
>  Issue Type: Improvement
>  Components: core
>Affects Versions: trunk
>Reporter: Taklon Stephen Wu
>  Labels: patch
> Fix For: trunk
>
> Attachments: OOZIE-2617-1.patch, OOZIE-2617-2.patch
>
>
> Current logic of acton configuration only READ xml files within Oozie action 
> directory, but for pig and other applications, properties files are supported 
> and may be default configuration file format. To simplify the logic of 
> reusing these properties files when using Oozie with them, we add properties 
> file reading in action conf.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2617) Read properties files in action configurations

[Taklon Stephen Wu (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15404824#comment-15404824
 ] 

Taklon Stephen Wu commented on OOZIE-2617:
--

sure, and another patch has been updated and uploaded, thanks for reviewing it.

> Read properties files in action configurations
> --
>
> Key: OOZIE-2617
> URL: https://issues.apache.org/jira/browse/OOZIE-2617
> Project: Oozie
>  Issue Type: Improvement
>  Components: core
>Affects Versions: trunk
>Reporter: Taklon Stephen Wu
>  Labels: patch
> Fix For: trunk
>
> Attachments: OOZIE-2617-1.patch, OOZIE-2617-2.patch
>
>
> Current logic of acton configuration only READ xml files within Oozie action 
> directory, but for pig and other applications, properties files are supported 
> and may be default configuration file format. To simplify the logic of 
> reusing these properties files when using Oozie with them, we add properties 
> file reading in action conf.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2516) Update web service documentation for jobs API

[Robert Kanter (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15404759#comment-15404759
 ] 

Robert Kanter commented on OOZIE-2516:
--

+1

> Update web service documentation for jobs API
> -
>
> Key: OOZIE-2516
> URL: https://issues.apache.org/jira/browse/OOZIE-2516
> Project: Oozie
>  Issue Type: Bug
>  Components: docs
>Reporter: abhishek bafna
>Assignee: abhishek bafna
> Fix For: 4.2.0, 4.3.0
>
> Attachments: OOZIE-2516-01.patch
>
>
> 1. Support for the 'group' filter is removed from the jobs API but the 
> WebService documentation is not updated for the same.
> 2. jobs API supports the 'id' filter which is also not documented in the web 
> services.
> https://oozie.apache.org/docs/4.2.0/WebServicesAPI.html#Jobs_Information
> From WorkflowsJobGetJPAExecutor and StoreStatusFilter
> {code}
> if (entry.getKey().equals(OozieClient.FILTER_GROUP)) {
>   XLog.getLog(getClass()).warn("Filter by 'group' is not supported anymore");
> }
> ...
> if (entry.getKey().equals(OozieClient.FILTER_ID)) {
> ...
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OOZIE-2497) Some tests fail on windows due to hard coded URIs

[Robert Kanter (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-2497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robert Kanter updated OOZIE-2497:
-
Summary: Some tests fail on windows due to hard coded URIs  (was: Testcase 
fails on windows with hard coded URIs)

> Some tests fail on windows due to hard coded URIs
> -
>
> Key: OOZIE-2497
> URL: https://issues.apache.org/jira/browse/OOZIE-2497
> Project: Oozie
>  Issue Type: Bug
>  Components: tests
> Environment: Windows
>Reporter: abhishek bafna
>Assignee: abhishek bafna
>  Labels: patch
> Attachments: OOZIE-2497-01.patch
>
>
> Some of the test cases have hard coded file URIs like "file://" which makes 
> them to fail in the windows environment.
> testcase:
> {noformat}
> org.apache.oozie.command.coord.TestCoordSubmitXCommand#testELdataIO_xsd_4
> org.apache.oozie.action.hadoop.TestMapReduceActionExecutor#testConfigDefaultPropsToAction
> org.apache.oozie.command.wf.TestSignalXCommand#testJoinFail
> {noformat}
> Exception message
> {noformat}
> testELdataIO_xsd_4
> java.net.URISyntaxException: Illegal character in authority at index 7: 
> file://D:\w\oozie\core\target\test-data\oozietests\org.apache.oozie.command.coord.TestCoordSubmitXCommand\testELdataIO_xsd_4\coordinator.xml
>   at java.net.URI$Parser.fail(URI.java:2827)
>   at java.net.URI$Parser.parseAuthority(URI.java:3165)
>   at java.net.URI$Parser.parseHierarchical(URI.java:3076)
>   at java.net.URI$Parser.parse(URI.java:3032)
>   at java.net.URI.(URI.java:595)
>   at 
> org.apache.oozie.command.coord.TestCoordSubmitXCommand.testELdataIO_xsd_4(TestCoordSubmitXCommand.java:576)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:601)
>   at junit.framework.TestCase.runTest(TestCase.java:168)
>   at junit.framework.TestCase.runBare(TestCase.java:134)
>   at junit.framework.TestResult$1.protect(TestResult.java:110)
>   at junit.framework.TestResult.runProtected(TestResult.java:128)
>   at junit.framework.TestResult.run(TestResult.java:113)
>   at junit.framework.TestCase.run(TestCase.java:124)
>   at junit.framework.TestSuite.runTest(TestSuite.java:243)
>   at junit.framework.TestSuite.run(TestSuite.java:238)
>   at 
> org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:83)
>   at org.junit.runners.Suite.runChild(Suite.java:128)
>   at org.junit.runners.Suite.runChild(Suite.java:24)
>   at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>   at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
>   at java.util.concurrent.FutureTask.run(FutureTask.java:166)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
>   at java.lang.Thread.run(Thread.java:722)
> testConfigDefaultPropsToAction
> org.apache.oozie.command.CommandException: E0803: IO error, Illegal character 
> in authority at index 7: 
> file://D:\w\oozie\sharelib\streaming\target\test-data\oozietests\org.apache.oozie.action.hadoop.TestMapReduceActionExecutor\testConfigDefaultPropsToAction\workflow.xml
>   at 
> org.apache.oozie.command.wf.SubmitXCommand.execute(SubmitXCommand.java:278)
>   at 
> org.apache.oozie.command.wf.SubmitXCommand.execute(SubmitXCommand.java:76)
>   at org.apache.oozie.command.XCommand.call(XCommand.java:286)
>   at 
> org.apache.oozie.action.hadoop.TestMapReduceActionExecutor.testConfigDefaultPropsToAction(TestMapReduceActionExecutor.java:139)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:601)
>   at junit.framework.TestCase.runTest(TestCase.java:168)
>   at junit.framework.TestCase.runBare(TestCase.java:134)
>   at junit.framework.TestResult$1.protect(TestResult.java:110)
>   at junit.framework.TestResult.runProtected(TestResult.java:128)
>   at junit.framework.TestResult.run(TestResult.java:113)
>   at junit.framework.TestCase.run(TestCase.java:124)
>   at 

[jira] [Commented] (OOZIE-2497) Testcase fails on windows with hard coded URIs

[Robert Kanter (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15404729#comment-15404729
 ] 

Robert Kanter commented on OOZIE-2497:
--

+1

> Testcase fails on windows with hard coded URIs
> --
>
> Key: OOZIE-2497
> URL: https://issues.apache.org/jira/browse/OOZIE-2497
> Project: Oozie
>  Issue Type: Bug
>  Components: tests
> Environment: Windows
>Reporter: abhishek bafna
>Assignee: abhishek bafna
>  Labels: patch
> Attachments: OOZIE-2497-01.patch
>
>
> Some of the test cases have hard coded file URIs like "file://" which makes 
> them to fail in the windows environment.
> testcase:
> {noformat}
> org.apache.oozie.command.coord.TestCoordSubmitXCommand#testELdataIO_xsd_4
> org.apache.oozie.action.hadoop.TestMapReduceActionExecutor#testConfigDefaultPropsToAction
> org.apache.oozie.command.wf.TestSignalXCommand#testJoinFail
> {noformat}
> Exception message
> {noformat}
> testELdataIO_xsd_4
> java.net.URISyntaxException: Illegal character in authority at index 7: 
> file://D:\w\oozie\core\target\test-data\oozietests\org.apache.oozie.command.coord.TestCoordSubmitXCommand\testELdataIO_xsd_4\coordinator.xml
>   at java.net.URI$Parser.fail(URI.java:2827)
>   at java.net.URI$Parser.parseAuthority(URI.java:3165)
>   at java.net.URI$Parser.parseHierarchical(URI.java:3076)
>   at java.net.URI$Parser.parse(URI.java:3032)
>   at java.net.URI.(URI.java:595)
>   at 
> org.apache.oozie.command.coord.TestCoordSubmitXCommand.testELdataIO_xsd_4(TestCoordSubmitXCommand.java:576)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:601)
>   at junit.framework.TestCase.runTest(TestCase.java:168)
>   at junit.framework.TestCase.runBare(TestCase.java:134)
>   at junit.framework.TestResult$1.protect(TestResult.java:110)
>   at junit.framework.TestResult.runProtected(TestResult.java:128)
>   at junit.framework.TestResult.run(TestResult.java:113)
>   at junit.framework.TestCase.run(TestCase.java:124)
>   at junit.framework.TestSuite.runTest(TestSuite.java:243)
>   at junit.framework.TestSuite.run(TestSuite.java:238)
>   at 
> org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:83)
>   at org.junit.runners.Suite.runChild(Suite.java:128)
>   at org.junit.runners.Suite.runChild(Suite.java:24)
>   at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>   at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
>   at java.util.concurrent.FutureTask.run(FutureTask.java:166)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
>   at java.lang.Thread.run(Thread.java:722)
> testConfigDefaultPropsToAction
> org.apache.oozie.command.CommandException: E0803: IO error, Illegal character 
> in authority at index 7: 
> file://D:\w\oozie\sharelib\streaming\target\test-data\oozietests\org.apache.oozie.action.hadoop.TestMapReduceActionExecutor\testConfigDefaultPropsToAction\workflow.xml
>   at 
> org.apache.oozie.command.wf.SubmitXCommand.execute(SubmitXCommand.java:278)
>   at 
> org.apache.oozie.command.wf.SubmitXCommand.execute(SubmitXCommand.java:76)
>   at org.apache.oozie.command.XCommand.call(XCommand.java:286)
>   at 
> org.apache.oozie.action.hadoop.TestMapReduceActionExecutor.testConfigDefaultPropsToAction(TestMapReduceActionExecutor.java:139)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:601)
>   at junit.framework.TestCase.runTest(TestCase.java:168)
>   at junit.framework.TestCase.runBare(TestCase.java:134)
>   at junit.framework.TestResult$1.protect(TestResult.java:110)
>   at junit.framework.TestResult.runProtected(TestResult.java:128)
>   at junit.framework.TestResult.run(TestResult.java:113)
>   at junit.framework.TestCase.run(TestCase.java:124)
>   at junit.framework.TestSuite.runTest(TestSuite.java:243)
>   at junit.framework.TestSuite.run(TestSuite.java:238)

[jira] [Commented] (OOZIE-2349) Method getCoordJobInfo(String jobId, String filter, int offset, int length, boolean desc) is not present in LocalOozieClientCoord

[Robert Kanter (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15404721#comment-15404721
 ] 

Robert Kanter commented on OOZIE-2349:
--

+1

> Method getCoordJobInfo(String jobId, String filter, int offset, int length, 
> boolean desc) is not present in LocalOozieClientCoord
> -
>
> Key: OOZIE-2349
> URL: https://issues.apache.org/jira/browse/OOZIE-2349
> Project: Oozie
>  Issue Type: Improvement
>Reporter: Narayan Periwal
>Assignee: Narayan Periwal
> Attachments: OOZIE-2349-v0.patch
>
>
> In LocalOozieClientCoord, method getCoordJobInfo(String jobId) is present, 
> but the corresponding overloaded method getCoordJobInfo(String jobId, String 
> filter, int offset, int length, boolean desc) is not present. Adding this 
> method in LocalOozieClientCoord will enable us to use this in the unit 
> testcases. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2172) ZooKeeper Security Tests failed with JVM IBM JAVA

[Robert Kanter (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15404717#comment-15404717
 ] 

Robert Kanter commented on OOZIE-2172:
--

+1

I haven't tested this with the IBM JDK (I'll assume that [~GauravPande] 
verified the changes with IBM), but I did make sure it doesn't break/change 
Oracle JDK behavior.

> ZooKeeper Security Tests failed with JVM IBM JAVA
> -
>
> Key: OOZIE-2172
> URL: https://issues.apache.org/jira/browse/OOZIE-2172
> Project: Oozie
>  Issue Type: Test
>  Components: HA
>Affects Versions: 4.1.0, 4.2.0
> Environment:  zookeeper-security-tests]# mvn -verion
> Apache Maven 3.2.5 (12a6b3acb947671f09b81f49094c53f426d8cea1; 
> 2014-12-14T12:29:23-05:00)
> Maven home: /opt/apache-maven-3.2.5
> Java version: 1.7.0, vendor: IBM Corporation
> Java home: /usr/lib/jvm/ibm-java-x86_64-71-pxa6470_27sr2fp10-20141218_02/jre
> Default locale: en_US, platform encoding: ISO-8859-1
> OS name: "linux", version: "3.10.0-229.el7.x86_64", arch: "amd64", family: 
> "unix"
>Reporter: pascal oliva
>Assignee: Gaurav Pande
>  Labels: ibm-jdk, jaas, oozie, patch, security, zookeeper
> Attachments: OOZIE-2172.patch
>
>
> ZooKeeper Security Tests failed by using IBM JAVA
> ---
> Concurrency config is parallel='classes', perCoreThreadCount=false, 
> threadCount=1, useUnlimitedThreads=false
> Running org.apache.oozie.util.TestZKUtilsWithSecurity
> Tests run: 2, Failures: 0, Errors: 2, Skipped: 0, Time elapsed: 36.493 sec 
> <<< FAILURE!
> Results :
> Tests in error:
>   testCheckAndSetACLs(org.apache.oozie.util.TestZKUtilsWithSecurity): 
> org.apache.hcatalog.common.HCatException : 9001 : Exception occurred while 
> processing HCat request : MetaException while creating HMS client. Cause : 
> MetaException(message:Could not connect to meta store using any of the URIs 
> provided. Most recent failure: 
> org.apache.thrift.transport.TTransportException: java.net.ConnectException: 
> Connection refused(..)
>   testNewUsingACLs(org.apache.oozie.util.TestZKUtilsWithSecurity): Could not 
> configure server because SASL configuration did not allow the  ZooKeeper 
> server to authenticate itself properly: 
> javax.security.auth.login.LoginException: Bad JAAS configuration: 
> unrecognized option: storeKey
> ..
> Traces On 
> target/./surefire-reports/org.apache.oozie.util.TestZKUtilsWithSecurity-output.txt
> at 
> com.google.common.cache.LocalCache$LocalManualCache$1.load(LocalCache.java:4767)
> at 
> com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3568)
> at 
> com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2350)
> at 
> com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2313)
> at 
> com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2228)
> at com.google.common.cache.LocalCache.get(LocalCache.java:3965)
> at 
> com.google.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4764)
> at 
> org.apache.hcatalog.common.HiveClientCache.getOrCreate(HiveClientCache.java:167)
> at 
> org.apache.hcatalog.common.HiveClientCache.get(HiveClientCache.java:143)
> at 
> org.apache.hcatalog.common.HCatUtil.getHiveClient(HCatUtil.java:548)
> at 
> org.apache.hcatalog.api.HCatClientHMSImpl.initialize(HCatClientHMSImpl.java:596)
> ... 27 more
> testNewUsingACLs(org.apache.oozie.util.TestZKUtilsWithSecurity)  Time 
> elapsed: 0.009 sec  <<< ERROR!
> java.io.IOException: Could not configure server because SASL configuration 
> did not allow the  ZooKeeper server to authenticate itself properly: 
> javax.security.auth.login.LoginException: Bad JAAS configuration: 
> unrecognized option: storeKey
> at 
> org.apache.zookeeper.server.ServerCnxnFactory.configureSaslLogin(ServerCnxnFactory.java:207)
> at 
> org.apache.zookeeper.server.NIOServerCnxnFactory.configure(NIOServerCnxnFactory.java:87)
> at 
> org.apache.zookeeper.server.ZooKeeperServerMain.runFromConfig(ZooKeeperServerMain.java:111)
> at 
> org.apache.curator.test.TestingZooKeeperMain.runFromConfig(TestingZooKeeperMain.java:73)
> at 
> org.apache.curator.test.TestingZooKeeperServer$1.run(TestingZooKeeperServer.java:134)
> at java.lang.Thread.run(Thread.java:863)
> ...
> howto reproduce :
> git clone -b branch-4.1 https://github.com/apache/oozie.git
> cd oozie
> ./bin/mkdistro.sh -DskipTests -Phadoop-2
> cd zookeeper-security-tests/
> mvn clean test -Phadoop-2



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2338) Invalid configuration defined reported for some valid configs

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15404711#comment-15404711
 ] 

Hadoop QA commented on OOZIE-2338:
--

Testing JIRA OOZIE-2338

Cleaning local git workspace



{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:red}-1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than 
132
.{color:red}-1{color} the patch does not add/modify any testcase
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT 
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc 
warnings
{color:red}-1 COMPILE{color}
.{color:red}-1{color} HEAD does not compile
.{color:red}-1{color} patch does not compile
.{color:green}+1{color} the patch does not seem to introduce new javac 
warnings
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA 
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:red}-1 TESTS{color}
.Tests run: 1792
.Tests failed: 27
.Tests errors: 75

.The patch failed the following testcases:

.  
testCoordMaterializeTriggerService3(org.apache.oozie.service.TestCoordMaterializeTriggerService)
.  
testRaceConditionWithOldXerces(org.apache.oozie.workflow.lite.TestLiteWorkflowAppParser)
.  
testParserDefaultNameNodeFail(org.apache.oozie.workflow.lite.TestLiteWorkflowAppParser)
.  
testParserDefaultJobTrackerFail(org.apache.oozie.workflow.lite.TestLiteWorkflowAppParser)
.  
testWfNoForkJoin(org.apache.oozie.workflow.lite.TestLiteWorkflowAppParser)
.  
testSimpleForkJoin(org.apache.oozie.workflow.lite.TestLiteWorkflowAppParser)
.  
testNestedForkJoin(org.apache.oozie.workflow.lite.TestLiteWorkflowAppParser)
.  
testForkJoinFailure(org.apache.oozie.workflow.lite.TestLiteWorkflowAppParser)
.  
testNestedForkJoinFailure(org.apache.oozie.workflow.lite.TestLiteWorkflowAppParser)
.  
testTransitionFailure1(org.apache.oozie.workflow.lite.TestLiteWorkflowAppParser)
.  testTransition2(org.apache.oozie.workflow.lite.TestLiteWorkflowAppParser)
.  testTransition3(org.apache.oozie.workflow.lite.TestLiteWorkflowAppParser)
.  
testErrorTransitionForkJoin(org.apache.oozie.workflow.lite.TestLiteWorkflowAppParser)
.  
testDecisionForkJoin(org.apache.oozie.workflow.lite.TestLiteWorkflowAppParser)
.  
testDecisionsToJoinForkJoin(org.apache.oozie.workflow.lite.TestLiteWorkflowAppParser)
.  
testDecisionsToKillForkJoin(org.apache.oozie.workflow.lite.TestLiteWorkflowAppParser)
.  
testDecisionForkJoinFailure(org.apache.oozie.workflow.lite.TestLiteWorkflowAppParser)
.  
testDecisionToEndForkJoinFailure(org.apache.oozie.workflow.lite.TestLiteWorkflowAppParser)
.  
testDecisionTwoPathsForkJoin(org.apache.oozie.workflow.lite.TestLiteWorkflowAppParser)
.  
testMultipleDecisionThreePathsForkJoin(org.apache.oozie.workflow.lite.TestLiteWorkflowAppParser)
.  
testMultipleDecisionThreePathsForkJoinFailure(org.apache.oozie.workflow.lite.TestLiteWorkflowAppParser)
.  
testMultipleDecisionThreePathsForkJoinFailure2(org.apache.oozie.workflow.lite.TestLiteWorkflowAppParser)
.  
testDecisionMultipleForks(org.apache.oozie.workflow.lite.TestLiteWorkflowAppParser)
.  
testForkJoinMismatch(org.apache.oozie.workflow.lite.TestLiteWorkflowAppParser)
.  
testForkJoinDuplicateTransitionsFromFork(org.apache.oozie.workflow.lite.TestLiteWorkflowAppParser)
.  
testDisableWFValidateForkJoin(org.apache.oozie.workflow.lite.TestLiteWorkflowAppParser)
.  
testCoordinatorActionCommandsSubmitAndStart(org.apache.oozie.sla.TestSLAEventGeneration)

{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch 


{color:red}*-1 Overall result, please check the reported -1(s)*{color}


The full output of the test-patch run is available at

.   https://builds.apache.org/job/oozie-trunk-precommit-build/3173/

> Invalid configuration defined reported for some valid configs
> -
>
> Key: OOZIE-2338
> URL: https://issues.apache.org/jira/browse/OOZIE-2338
> Project: Oozie
>  Issue Type: Bug
>  Components: core
>Affects Versions: 4.2.0
>Reporter: Robert Kanter
>Assignee: Robert Kanter
> Attachments: OOZIE-2338.001.patch, OOZIE-2338.002.patch
>
>
> OOZIE-1890 moved all default config values from 

[jira] [Updated] (OOZIE-2172) ZooKeeper Security Tests failed with JVM IBM JAVA

[Robert Kanter (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-2172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robert Kanter updated OOZIE-2172:
-
Assignee: Gaurav Pande

> ZooKeeper Security Tests failed with JVM IBM JAVA
> -
>
> Key: OOZIE-2172
> URL: https://issues.apache.org/jira/browse/OOZIE-2172
> Project: Oozie
>  Issue Type: Test
>  Components: HA
>Affects Versions: 4.1.0, 4.2.0
> Environment:  zookeeper-security-tests]# mvn -verion
> Apache Maven 3.2.5 (12a6b3acb947671f09b81f49094c53f426d8cea1; 
> 2014-12-14T12:29:23-05:00)
> Maven home: /opt/apache-maven-3.2.5
> Java version: 1.7.0, vendor: IBM Corporation
> Java home: /usr/lib/jvm/ibm-java-x86_64-71-pxa6470_27sr2fp10-20141218_02/jre
> Default locale: en_US, platform encoding: ISO-8859-1
> OS name: "linux", version: "3.10.0-229.el7.x86_64", arch: "amd64", family: 
> "unix"
>Reporter: pascal oliva
>Assignee: Gaurav Pande
>  Labels: ibm-jdk, jaas, oozie, patch, security, zookeeper
> Attachments: OOZIE-2172.patch
>
>
> ZooKeeper Security Tests failed by using IBM JAVA
> ---
> Concurrency config is parallel='classes', perCoreThreadCount=false, 
> threadCount=1, useUnlimitedThreads=false
> Running org.apache.oozie.util.TestZKUtilsWithSecurity
> Tests run: 2, Failures: 0, Errors: 2, Skipped: 0, Time elapsed: 36.493 sec 
> <<< FAILURE!
> Results :
> Tests in error:
>   testCheckAndSetACLs(org.apache.oozie.util.TestZKUtilsWithSecurity): 
> org.apache.hcatalog.common.HCatException : 9001 : Exception occurred while 
> processing HCat request : MetaException while creating HMS client. Cause : 
> MetaException(message:Could not connect to meta store using any of the URIs 
> provided. Most recent failure: 
> org.apache.thrift.transport.TTransportException: java.net.ConnectException: 
> Connection refused(..)
>   testNewUsingACLs(org.apache.oozie.util.TestZKUtilsWithSecurity): Could not 
> configure server because SASL configuration did not allow the  ZooKeeper 
> server to authenticate itself properly: 
> javax.security.auth.login.LoginException: Bad JAAS configuration: 
> unrecognized option: storeKey
> ..
> Traces On 
> target/./surefire-reports/org.apache.oozie.util.TestZKUtilsWithSecurity-output.txt
> at 
> com.google.common.cache.LocalCache$LocalManualCache$1.load(LocalCache.java:4767)
> at 
> com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3568)
> at 
> com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2350)
> at 
> com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2313)
> at 
> com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2228)
> at com.google.common.cache.LocalCache.get(LocalCache.java:3965)
> at 
> com.google.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4764)
> at 
> org.apache.hcatalog.common.HiveClientCache.getOrCreate(HiveClientCache.java:167)
> at 
> org.apache.hcatalog.common.HiveClientCache.get(HiveClientCache.java:143)
> at 
> org.apache.hcatalog.common.HCatUtil.getHiveClient(HCatUtil.java:548)
> at 
> org.apache.hcatalog.api.HCatClientHMSImpl.initialize(HCatClientHMSImpl.java:596)
> ... 27 more
> testNewUsingACLs(org.apache.oozie.util.TestZKUtilsWithSecurity)  Time 
> elapsed: 0.009 sec  <<< ERROR!
> java.io.IOException: Could not configure server because SASL configuration 
> did not allow the  ZooKeeper server to authenticate itself properly: 
> javax.security.auth.login.LoginException: Bad JAAS configuration: 
> unrecognized option: storeKey
> at 
> org.apache.zookeeper.server.ServerCnxnFactory.configureSaslLogin(ServerCnxnFactory.java:207)
> at 
> org.apache.zookeeper.server.NIOServerCnxnFactory.configure(NIOServerCnxnFactory.java:87)
> at 
> org.apache.zookeeper.server.ZooKeeperServerMain.runFromConfig(ZooKeeperServerMain.java:111)
> at 
> org.apache.curator.test.TestingZooKeeperMain.runFromConfig(TestingZooKeeperMain.java:73)
> at 
> org.apache.curator.test.TestingZooKeeperServer$1.run(TestingZooKeeperServer.java:134)
> at java.lang.Thread.run(Thread.java:863)
> ...
> howto reproduce :
> git clone -b branch-4.1 https://github.com/apache/oozie.git
> cd oozie
> ./bin/mkdistro.sh -DskipTests -Phadoop-2
> cd zookeeper-security-tests/
> mvn clean test -Phadoop-2



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Build failed in Jenkins: oozie-trunk-precommit-build #3173

[Apache Jenkins Server]

See 

Changes:

[jvishwakarma] OOZIE-2440 Exponential re-try policy for workflow action 
(satishsaley

--
[...truncated 8270 lines...]
[INFO] 
[INFO] --- maven-resources-plugin:2.5:resources (default-resources) @ 
oozie-mini ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:compile (default-compile) @ oozie-mini 
---
[INFO] No sources to compile
[INFO] 
[INFO] --- maven-resources-plugin:2.5:testResources (default-testResources) @ 
oozie-mini ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 3 resources
[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:testCompile (default-testCompile) @ 
oozie-mini ---
[INFO] Nothing to compile - all classes are up to date
[INFO] 
[INFO] --- maven-surefire-plugin:2.12.2:test (default-test) @ oozie-mini ---
[INFO] Tests are skipped.
[INFO] 
[INFO] --- maven-jar-plugin:2.3.1:jar (default-jar) @ oozie-mini ---
[INFO] Building jar: 

[INFO] 
[INFO] --- maven-site-plugin:2.0-beta-6:attach-descriptor (attach-descriptor) @ 
oozie-mini ---
[INFO] 
[INFO] --- maven-assembly-plugin:2.2.1:single (default-cli) @ oozie-mini ---
[INFO] Reading assembly descriptor: src/main/assemblies/empty.xml
[INFO] 
[INFO] 
[INFO] Building Apache Oozie Distro 4.3.0-SNAPSHOT
[INFO] 
[INFO] 
[INFO] --- maven-remote-resources-plugin:1.5:process (default) @ oozie-distro 
---
[INFO] 
[INFO] --- maven-resources-plugin:2.5:resources (default-resources) @ 
oozie-distro ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:compile (default-compile) @ oozie-distro 
---
[INFO] No sources to compile
[INFO] 
[INFO] --- maven-resources-plugin:2.5:testResources (default-testResources) @ 
oozie-distro ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:testCompile (default-testCompile) @ 
oozie-distro ---
[INFO] No sources to compile
[INFO] 
[INFO] --- maven-surefire-plugin:2.12.2:test (default-test) @ oozie-distro ---
[INFO] Tests are skipped.
[INFO] 
[INFO] --- maven-jar-plugin:2.3.1:jar (default-jar) @ oozie-distro ---
[INFO] Building jar: 

[INFO] 
[INFO] --- maven-site-plugin:2.0-beta-6:attach-descriptor (attach-descriptor) @ 
oozie-distro ---
[INFO] 
[INFO] --- maven-antrun-plugin:1.6:run (default) @ oozie-distro ---
[INFO] Executing tasks

main:
  [get] Getting: 
http://archive.apache.org/dist/tomcat/tomcat-6/v6.0.44/bin/apache-tomcat-6.0.44.tar.gz
  [get] To: 

.
[mkdir] Created dir: 

   [gunzip] Expanding 

 to 

[untar] Expanding: 

 into 

   [delete] Deleting directory 

[mkdir] Created dir: 

   [delete] Deleting: 

[jira] [Commented] (OOZIE-2566) TestCoordActionInputCheckXCommand.testCoordActionInputCheckXCommandUniqueness() is flaky

[Peter Bacsko (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15404702#comment-15404702
 ] 

Peter Bacsko commented on OOZIE-2566:
-

[~rkanter], [~puru], [~rohini], [~abhishekbafna] - any ideas regarding this 
problem? Testcase or oozie-core issue?

> TestCoordActionInputCheckXCommand.testCoordActionInputCheckXCommandUniqueness()
>  is flaky
> 
>
> Key: OOZIE-2566
> URL: https://issues.apache.org/jira/browse/OOZIE-2566
> Project: Oozie
>  Issue Type: Bug
>  Components: core
>Reporter: Peter Bacsko
>Assignee: Peter Bacsko
>
> The testcase testCoordActionInputCheckXCommandUniqueness is unstable.
> We add three XCommands with the same actionId (entityKeys are different) into 
> the CallableQueueService. Only the first XCommand is expected to run.
> The reason why sometimes either the 2nd or 3rd XCommand executes is because 
> as soon as the first starts to run, its removed from the {{uniqueCallables}} 
> map immediately. If the first scheduled task runs quickly, then either the 
> 2nd or 3rd XCommand has the chance to get scheduled.
> Step by step:
> 1. Schedule first XCommand
> 2. XCommand is added to {{uniqueCallables}}
> 3. Schedule second XCommand
> 4. First XCommand starts to run in the thread pool and removes itself from 
> {{uniqueCallables}} (see {{CallableWrapper.run()}})
> 5. Second XCommand can successfully add itself to {{uniqueCallables}}
> 6. Second XCommand starts to run
> Please clarify whether this is the expected behavior of CallableQueueService.
> If not, then moving {{removeFromUniqueCallables()}} to the finally block 
> solves the problem.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2555) Oozie SSL enable setup does not return port for admin -servers

[Robert Kanter (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15404707#comment-15404707
 ] 

Robert Kanter commented on OOZIE-2555:
--

+1

> Oozie SSL enable setup does not return port for admin -servers
> --
>
> Key: OOZIE-2555
> URL: https://issues.apache.org/jira/browse/OOZIE-2555
> Project: Oozie
>  Issue Type: Bug
>  Components: core
>Reporter: abhishek bafna
>Assignee: abhishek bafna
> Attachments: OOZIE-2555-00.patch
>
>
> When oozie is setup with SSL. The *oozie admin -servers -oozie 
> https://hostname:11443/oozie* is not returning the port in output.
> {noformat}
> [oozie@host oozie]$ oozie admin -servers -oozie https://hostname:11443/oozie
> hostname : https://hostname:/oozie
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2572) SLA DURATION miss not shown when job is running for longer than expected time

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15404607#comment-15404607
 ] 

Hadoop QA commented on OOZIE-2572:
--

Testing JIRA OOZIE-2572

Cleaning local git workspace



{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:green}+1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than 
132
.{color:green}+1{color} the patch does adds/modifies 1 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT 
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc 
warnings
{color:red}-1 COMPILE{color}
.{color:red}-1{color} HEAD does not compile
.{color:red}-1{color} patch does not compile
.{color:green}+1{color} the patch does not seem to introduce new javac 
warnings
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA 
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:red}-1 TESTS{color} - patch does not compile, cannot run testcases
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch 


{color:red}*-1 Overall result, please check the reported -1(s)*{color}


The full output of the test-patch run is available at

.   https://builds.apache.org/job/oozie-trunk-precommit-build/3172/

> SLA DURATION miss not shown when job is running for longer than expected time
> -
>
> Key: OOZIE-2572
> URL: https://issues.apache.org/jira/browse/OOZIE-2572
> Project: Oozie
>  Issue Type: Bug
>Reporter: Satish Subhashrao Saley
>Assignee: Satish Subhashrao Saley
> Attachments: OOZIE-2572-1.patch, OOZIE-2572-2.patch, 
> OOZIE-2572-3.patch, OOZIE-2572-4.patch, OOZIE-2572-5.patch
>
>
> If job is running for beyond expected duration, Oozie UI is not slowing SLA 
> for duration miss.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Build failed in Jenkins: oozie-trunk-precommit-build #3172

[Apache Jenkins Server]

See 

--
[...truncated 8114 lines...]
[INFO] --- maven-compiler-plugin:2.3.2:compile (default-compile) @ oozie-tools 
---
[INFO] Nothing to compile - all classes are up to date
[INFO] 
[INFO] --- maven-resources-plugin:2.5:testResources (default-testResources) @ 
oozie-tools ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 2 resources
[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:testCompile (default-testCompile) @ 
oozie-tools ---
[INFO] Nothing to compile - all classes are up to date
[INFO] 
[INFO] --- maven-surefire-plugin:2.12.2:test (default-test) @ oozie-tools ---
[INFO] Tests are skipped.
[INFO] 
[INFO] --- maven-jar-plugin:2.3.1:jar (default-jar) @ oozie-tools ---
[INFO] Building jar: 

[INFO] 
[INFO] --- maven-site-plugin:2.0-beta-6:attach-descriptor (attach-descriptor) @ 
oozie-tools ---
[INFO] 
[INFO] --- maven-assembly-plugin:2.2.1:single (default-cli) @ oozie-tools ---
[INFO] Reading assembly descriptor: ../src/main/assemblies/tools.xml
[WARNING] The following patterns were never triggered in this artifact 
exclusion filter:
o  '*:*:pom:*'

[INFO] Copying files to 

[WARNING] Assembly file: 

 is not a regular file (it may be a directory). It cannot be attached to the 
project build for installation or deployment.
[INFO] 
[INFO] 
[INFO] Building Apache Oozie MiniOozie 4.3.0-SNAPSHOT
[INFO] 
[INFO] 
[INFO] --- maven-remote-resources-plugin:1.5:process (default) @ oozie-mini ---
[INFO] 
[INFO] --- maven-resources-plugin:2.5:resources (default-resources) @ 
oozie-mini ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:compile (default-compile) @ oozie-mini 
---
[INFO] No sources to compile
[INFO] 
[INFO] --- maven-resources-plugin:2.5:testResources (default-testResources) @ 
oozie-mini ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 3 resources
[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:testCompile (default-testCompile) @ 
oozie-mini ---
[INFO] Nothing to compile - all classes are up to date
[INFO] 
[INFO] --- maven-surefire-plugin:2.12.2:test (default-test) @ oozie-mini ---
[INFO] Tests are skipped.
[INFO] 
[INFO] --- maven-jar-plugin:2.3.1:jar (default-jar) @ oozie-mini ---
[INFO] Building jar: 

[INFO] 
[INFO] --- maven-site-plugin:2.0-beta-6:attach-descriptor (attach-descriptor) @ 
oozie-mini ---
[INFO] 
[INFO] --- maven-assembly-plugin:2.2.1:single (default-cli) @ oozie-mini ---
[INFO] Reading assembly descriptor: src/main/assemblies/empty.xml
[INFO] 
[INFO] 
[INFO] Building Apache Oozie Distro 4.3.0-SNAPSHOT
[INFO] 
[INFO] 
[INFO] --- maven-remote-resources-plugin:1.5:process (default) @ oozie-distro 
---
[INFO] 
[INFO] --- maven-resources-plugin:2.5:resources (default-resources) @ 
oozie-distro ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:compile (default-compile) @ oozie-distro 
---
[INFO] No sources to compile
[INFO] 
[INFO] --- maven-resources-plugin:2.5:testResources (default-testResources) @ 
oozie-distro ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:testCompile (default-testCompile) @ 
oozie-distro ---
[INFO] No sources to compile
[INFO] 
[INFO] --- maven-surefire-plugin:2.12.2:test (default-test) @ oozie-distro ---

[jira] [Commented] (OOZIE-2342) Coordinator docs still mention old SLA system

[Rohini Palaniswamy (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15404603#comment-15404603
 ] 

Rohini Palaniswamy commented on OOZIE-2342:
---

[~satishsaley],
   Can you pick this up and OOZIE-2364? It would be good to clean this up for 
Oozie 4.3 itself

> Coordinator docs still mention old SLA system
> -
>
> Key: OOZIE-2342
> URL: https://issues.apache.org/jira/browse/OOZIE-2342
> Project: Oozie
>  Issue Type: Sub-task
>  Components: docs
>Affects Versions: trunk
>Reporter: Robert Kanter
> Fix For: trunk
>
>
> The Coordinator docs page still mentions the old SLA system, which requires 
> the Yahoo! grid thing.  
> We should do one of the following:
> # Replace this with updated examples that use the new SLA system
> # Add a big note that this is deprecated and link to the newer SLA docs page
> # Remove the section; we already document the new SLA system elsewhere.
> http://oozie.apache.org/docs/4.2.0/CoordinatorFunctionalSpec.html#a12._SLA_Handling



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2585) Disable or remove test case TestPartitionDependencyManagerEhcache.testMemoryUsageAndSpeedOverflowToDisk and testMemoryUsageAndSpeed

[Robert Kanter (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15404594#comment-15404594
 ] 

Robert Kanter commented on OOZIE-2585:
--

ping [~rohini]

> Disable or remove test case 
> TestPartitionDependencyManagerEhcache.testMemoryUsageAndSpeedOverflowToDisk 
> and testMemoryUsageAndSpeed
> ---
>
> Key: OOZIE-2585
> URL: https://issues.apache.org/jira/browse/OOZIE-2585
> Project: Oozie
>  Issue Type: Bug
>Reporter: Peter Bacsko
>Assignee: Peter Bacsko
>Priority: Minor
> Attachments: OOZIE-2585-001.patch, OOZIE-2585-002.patch
>
>
> There is a problem with the test case testMemoryUsageAndSpeedOverflowToDisk.
> After 20k entries, data is spilled to the disk. However, Ehcache writes on a 
> separate thread asynchronously and all put operations return immediately. The 
> problem is that after storing the 60k entries, we try to read them back 
> instantly, with many of them are still being flushed to the storage. As long 
> as disk write in progress, these entries are unavailable.
> I didn't find a reliable way to wait for disk writes. Therefore I suggest 
> disabling this test or eliminate it altogether -- it's a performance test 
> that runs on all kinds of machines during a build, I don't think it makes too 
> much sense to keep it.
> Also testMemoryUsageAndSpeed might be prone to test failures too - these two 
> assertions might evaluate to false:
> {code}
> assertTrue((endTime - startTime) < insertTimeinMillis);
> ...
> assertTrue((System.currentTimeMillis() - endTime) < retrievalTimeinMillis);
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2530) Remove jline from parent and sharelib pig pom

[Robert Kanter (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15404530#comment-15404530
 ] 

Robert Kanter commented on OOZIE-2530:
--

As I said before, we have multiple components, needing different versions of 
jline.  To allow for this, we need to figure out the correct versions for each 
component and set those in the sharelibs.  It seems like Pig isn't pulling in 
jline as a transitive dependency, which is unfortunate, as it makes this more 
of a burden on Oozie.

Looking at the current versions of jline after building Oozie, I see:
{noformat}
./sharelib/target/oozie-sharelib-4.3.0-SNAPSHOT/share/lib/hive/jline-0.9.94.jar
./sharelib/target/oozie-sharelib-4.3.0-SNAPSHOT/share/lib/hive2/jline-0.9.94.jar
./sharelib/target/oozie-sharelib-4.3.0-SNAPSHOT/share/lib/pig/jline-0.9.94.jar
./sharelib/target/oozie-sharelib-4.3.0-SNAPSHOT/share/lib/spark/jline-0.9.94.jar
./sharelib/target/oozie-sharelib-4.3.0-SNAPSHOT/share/lib/spark/jline-2.10.5.jar
./webapp/target/oozie-webapp-4.3.0-SNAPSHOT/WEB-INF/lib/jline-0.9.94.jar
{noformat}
As it stands, Everything (by default) is using jline 0.9.94, though Spark 
somehow has two versions of it.  Given that, I'd say that the correct thing 
here would be to do something like OOZIE-2409, which specified a different 
version for Hive.  I think what we should do here is to leave jline to 0.9.94 
as-is, add a new {{hive.jline.version}} property in the root pom, set to 0.9.94 
by default for now (we can change it later if we update the minimum Hive 
version), and have the Hive sharelibs use {{hive.jline.version}}.  That's the 
same way we handle a newer guava version for Spark (see 
{{spark.guava.version}}).
Sounds good?

> Remove jline from parent and sharelib pig pom
> -
>
> Key: OOZIE-2530
> URL: https://issues.apache.org/jira/browse/OOZIE-2530
> Project: Oozie
>  Issue Type: Improvement
>Affects Versions: trunk
>Reporter: Thomas Poepping
> Attachments: OOZIE-2530.01.patch, OOZIE-2530.patch
>
>
> This patch removes an unnecessary hardcoded dependency on jline in the oozie 
> parent pom and pig sharelib pom.
> The jline dependencies were required before OOZIE-704, which removed the 
> direct dependencies on jline in oozie code. Since that patch, the direct 
> dependency on jline in oozie has been unnecessary. This will also fix 
> OOZIE-2409.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OOZIE-2338) Invalid configuration defined reported for some valid configs

[Robert Kanter (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-2338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robert Kanter updated OOZIE-2338:
-
Attachment: OOZIE-2338.002.patch

> Invalid configuration defined reported for some valid configs
> -
>
> Key: OOZIE-2338
> URL: https://issues.apache.org/jira/browse/OOZIE-2338
> Project: Oozie
>  Issue Type: Bug
>  Components: core
>Affects Versions: 4.2.0
>Reporter: Robert Kanter
>Assignee: Robert Kanter
> Attachments: OOZIE-2338.001.patch, OOZIE-2338.002.patch
>
>
> OOZIE-1890 moved all default config values from the code and oozie-site into 
> oozie-default; it also added a WARN message on startup when a config is found 
> in oozie-site that's not defined in oozie-default.  
> However, this can produce some false positives.  In particular, we observed 
> these:
> {noformat}
> 2015-08-18 01:16:32,365 WARN org.apache.oozie.service.ConfigurationService: 
> SERVER[] Invalid configuration defined, [oozie.email.smtp.auth]
> 2015-08-18 01:16:32,365 WARN org.apache.oozie.service.ConfigurationService: 
> SERVER[] Invalid configuration defined, [oozie.email.smtp.host]
> 2015-08-18 01:16:32,365 WARN org.apache.oozie.service.ConfigurationService: 
> SERVER[] Invalid configuration defined, 
> [oozie.service.HCatAccessorService.hcat.configuration]
> 2015-08-18 01:16:32,365 WARN org.apache.oozie.service.ConfigurationService: 
> SERVER[] Invalid configuration defined, [oozie.email.from.address]
> 2015-08-18 01:16:32,366 WARN org.apache.oozie.service.ConfigurationService: 
> SERVER[] Invalid configuration defined, 
> [oozie.service.ProxyUserService.proxyuser.hue.hosts]
> 2015-08-18 01:16:32,366 WARN org.apache.oozie.service.ConfigurationService: 
> SERVER[] Invalid configuration defined, 
> [oozie.service.GroupsService.hadoop.security.group.mapping]
> 2015-08-18 01:16:32,366 WARN org.apache.oozie.service.ConfigurationService: 
> SERVER[] Invalid configuration defined, [oozie.email.smtp.port]
> 2015-08-18 01:16:32,366 WARN org.apache.oozie.service.ConfigurationService: 
> SERVER[] Invalid configuration defined, 
> [oozie.service.ProxyUserService.proxyuser.hue.groups]
> {noformat}
> While harmful, these messages may be concerning or confusing for users.
> Some of these are simply missing from oozie-default (because they have no 
> default value?), while others such as the ProxyUserService configs, will 
> require special handling because the property name can vary.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OOZIE-2572) SLA DURATION miss not shown when job is running for longer than expected time

[Satish Subhashrao Saley (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-2572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Satish Subhashrao Saley updated OOZIE-2572:
---
Attachment: OOZIE-2572-5.patch

- Reverting back the changes made to JSON response. Now keeping JSON response 
as it is.
- Actual duration for running job should be in milliseconds
- Corrections in the documentation

> SLA DURATION miss not shown when job is running for longer than expected time
> -
>
> Key: OOZIE-2572
> URL: https://issues.apache.org/jira/browse/OOZIE-2572
> Project: Oozie
>  Issue Type: Bug
>Reporter: Satish Subhashrao Saley
>Assignee: Satish Subhashrao Saley
> Attachments: OOZIE-2572-1.patch, OOZIE-2572-2.patch, 
> OOZIE-2572-3.patch, OOZIE-2572-4.patch, OOZIE-2572-5.patch
>
>
> If job is running for beyond expected duration, Oozie UI is not slowing SLA 
> for duration miss.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2618) Include hive-common in hive-beeline as compile component for hive-server2 (hive2 action)

[Peter Cseh (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2618?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15404341#comment-15404341
 ] 

Peter Cseh commented on OOZIE-2618:
---

+1 (nonbinding) 
Thank you for the patch!
OOZIE-2613 might be blocked by this issue.


> Include hive-common in hive-beeline as compile component for hive-server2 
> (hive2 action)
> 
>
> Key: OOZIE-2618
> URL: https://issues.apache.org/jira/browse/OOZIE-2618
> Project: Oozie
>  Issue Type: Improvement
>  Components: action
>Affects Versions: trunk
>Reporter: Taklon Stephen Wu
>  Labels: maven, patch
> Fix For: trunk
>
> Attachments: OOZIE-2618-1.patch
>
>
> OOZIE-1876 has excluded hive-common in hive-beeline and isn’t compatible with 
> hive version which is higher than 0.13.1. This change adds hive-common as 
> part of the dependency for hive-beeline and submodules of it can be used 
> beeline correctly.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2617) Read properties files in action configurations

[Peter Cseh (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15404327#comment-15404327
 ] 

Peter Cseh commented on OOZIE-2617:
---

Thank you for the contribution [~wutak...@amazon.com]! 
Cloud you update {{oozie/docs/src/site/twiki/AG_ActionConfiguration.twiki}} 
with the new functionality?


> Read properties files in action configurations
> --
>
> Key: OOZIE-2617
> URL: https://issues.apache.org/jira/browse/OOZIE-2617
> Project: Oozie
>  Issue Type: Improvement
>  Components: core
>Affects Versions: trunk
>Reporter: Taklon Stephen Wu
>  Labels: patch
> Fix For: trunk
>
> Attachments: OOZIE-2617-1.patch
>
>
> Current logic of acton configuration only READ xml files within Oozie action 
> directory, but for pig and other applications, properties files are supported 
> and may be default configuration file format. To simplify the logic of 
> reusing these properties files when using Oozie with them, we add properties 
> file reading in action conf.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2588) Support getting credentials for same cluster hcat when credentials config is empty

[Peter Cseh (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15404300#comment-15404300
 ] 

Peter Cseh commented on OOZIE-2588:
---

Can you move the powermock dependency to the root pom.xml and just reference it 
from the core/pom.xml? 
It would make usage and version bumps easier in the future.


> Support getting credentials for same cluster hcat when credentials config is 
> empty
> --
>
> Key: OOZIE-2588
> URL: https://issues.apache.org/jira/browse/OOZIE-2588
> Project: Oozie
>  Issue Type: Bug
>Reporter: Satish Subhashrao Saley
>Assignee: Satish Subhashrao Saley
>Priority: Minor
> Attachments: OOZIE-2588-1.patch, OOZIE-2588-2.patch, 
> OOZIE-2588-3.patch, OOZIE-2588-4.patch, OOZIE-2588-5.patch, 
> OOZIE-2588-6.patch, OOZIE-2588-7.patch
>
>
> Oozie should get credentials for the same cluster hcat when 
> hcat.metastore.uri and hcat.metastore.principal are not specified. It will 
> eliminate lot of trouble for the users.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2616) Add Tez profile for Hive and Pig sharelibs

[Peter Cseh (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15404223#comment-15404223
 ] 

Peter Cseh commented on OOZIE-2616:
---

Thank you for the contribution [~poeppt]!
Could you please provide a Hadoop, Pig and Hive version when Oozie compiles 
with active Tez profile?


> Add Tez profile for Hive and Pig sharelibs
> --
>
> Key: OOZIE-2616
> URL: https://issues.apache.org/jira/browse/OOZIE-2616
> Project: Oozie
>  Issue Type: Improvement
>Affects Versions: trunk
>Reporter: Thomas Poepping
> Attachments: OOZIE-2616-1.patch
>
>
> More recent versions of Hive and Pig depend on Tez as their default execution 
> engine. As the sharelib for those applications stands today, those 
> applications are unable to run Tez jobs. This patch adds an optional profile 
> to the root pom that will include the proper Tez dependencies into those 
> sharelibs, while excluding problematic jars to keep the sharelibs from having 
> classpath duplication issues. *Note*: this patch will only be required for 
> those clients who specify their own, newer versions of Hive or Pig. This 
> change is not required for the current supported versions of those 
> applications (hence why the profile is off by default).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Jenkins build is back to normal : oozie-trunk-find-patches-available #128183

[Apache Jenkins Server]

See 

Build failed in Jenkins: oozie-trunk-find-patches-available #128182

[Apache Jenkins Server]

See 

--
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on H1 (Mapreduce Hadoop Pig Hdfs yahoo-not-h2) in workspace 

 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url 
 > https://git-wip-us.apache.org/repos/asf/oozie.git # timeout=10
Fetching upstream changes from https://git-wip-us.apache.org/repos/asf/oozie.git
 > git --version # timeout=10
 > git -c core.askpass=true fetch --tags --progress 
 > https://git-wip-us.apache.org/repos/asf/oozie.git 
 > +refs/heads/*:refs/remotes/origin/*
 > git rev-parse refs/remotes/origin/master^{commit} # timeout=10
 > git rev-parse refs/remotes/origin/origin/master^{commit} # timeout=10
Checking out Revision 2730030624b46e0d6ffe869f7ccf3207173681e3 
(refs/remotes/origin/master)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 2730030624b46e0d6ffe869f7ccf3207173681e3
 > git rev-list 2730030624b46e0d6ffe869f7ccf3207173681e3 # timeout=10
[oozie-trunk-find-patches-available] $ /bin/bash 
/tmp/hudson1144535053035692146.sh
mkdir: cannot create directory 
':
 File exists
  % Total% Received % Xferd  Average Speed   TimeTime Time  Current
 Dload  Upload   Total   SpentLeft  Speed

  0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 0
  0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 0
100  227k0  227k0 0   212k  0 --:--:--  0:00:01 --:--:--  212k
100  561k0  561k0 0   270k  0 --:--:--  0:00:02 --:--:--  270k
100  965k0  965k0 0   301k  0 --:--:--  0:00:03 --:--:--  301k
100 1264k0 1264k0 0   309k  0 --:--:--  0:00:04 --:--:--  309k
100 1571k0 1571k0 0   306k  0 --:--:--  0:00:05 --:--:--  311k
100 1571k0 1571k0 0   306k  0 --:--:--  0:00:05 --:--:--  331k
curl: (18) transfer closed with outstanding read data remaining
Could not retrieve available patches from JIRA
Build step 'Execute shell' marked build as failure

[jira] Subscription: Oozie Patch Available

[jira]

Issue Subscription
Filter: Oozie Patch Available (95 issues)

Subscriber: ooziedaily

Key Summary
OOZIE-2623  Oozie should use a dummy OutputFormat
https://issues.apache.org/jira/browse/OOZIE-2623
OOZIE-2618  Include hive-common in hive-beeline as compile component for 
hive-server2 (hive2 action)
https://issues.apache.org/jira/browse/OOZIE-2618
OOZIE-2617  Read properties files in action configurations
https://issues.apache.org/jira/browse/OOZIE-2617
OOZIE-2616  Add Tez profile for Hive and Pig sharelibs
https://issues.apache.org/jira/browse/OOZIE-2616
OOZIE-2606  Set spark.yarn.jars to fix Spark 2.0 with Oozie
https://issues.apache.org/jira/browse/OOZIE-2606
OOZIE-2589  CompletedActionXCommand priority param is of no effect
https://issues.apache.org/jira/browse/OOZIE-2589
OOZIE-2588  Support getting credentials for same cluster hcat when credentials 
config is empty
https://issues.apache.org/jira/browse/OOZIE-2588
OOZIE-2585  Disable or remove test case 
TestPartitionDependencyManagerEhcache.testMemoryUsageAndSpeedOverflowToDisk and 
testMemoryUsageAndSpeed
https://issues.apache.org/jira/browse/OOZIE-2585
OOZIE-2584  Eliminate Thread.sleep() calls in TestMemoryLocks
https://issues.apache.org/jira/browse/OOZIE-2584
OOZIE-2583  oozie throws EL Exception when reference variable name containing 
dot
https://issues.apache.org/jira/browse/OOZIE-2583
OOZIE-2582  Populating external child Ids for action failures
https://issues.apache.org/jira/browse/OOZIE-2582
OOZIE-2574  Oozie to support replication-enabled mysql urls
https://issues.apache.org/jira/browse/OOZIE-2574
OOZIE-2573  dataset url  which contains spaces can not be handled rightly
https://issues.apache.org/jira/browse/OOZIE-2573
OOZIE-2572  SLA DURATION miss not shown when job is running for longer than 
expected time
https://issues.apache.org/jira/browse/OOZIE-2572
OOZIE-2571  Add scala.binary.version Maven property so that Scala 2.11 can be 
used
https://issues.apache.org/jira/browse/OOZIE-2571
OOZIE-2569  Adding yarn-site, core-site, hdfs-site and mapred-site into spark 
launcher 
https://issues.apache.org/jira/browse/OOZIE-2569
OOZIE-2568  SSH  action pretends to retry automaticly when it failed
https://issues.apache.org/jira/browse/OOZIE-2568
OOZIE-2564  Create new log4j config for unit tests so that logged messages 
contain threads
https://issues.apache.org/jira/browse/OOZIE-2564
OOZIE-2555  Oozie SSL enable setup does not return port for admin -servers
https://issues.apache.org/jira/browse/OOZIE-2555
OOZIE-2552  Update ActiveMQ version for security and other fixes
https://issues.apache.org/jira/browse/OOZIE-2552
OOZIE-2538  Update HttpClient versions to close security vulnerabilities
https://issues.apache.org/jira/browse/OOZIE-2538
OOZIE-2534  Versioned action libs (similar to how SharedLibs works)
https://issues.apache.org/jira/browse/OOZIE-2534
OOZIE-2530  Remove jline from parent and sharelib pig pom
https://issues.apache.org/jira/browse/OOZIE-2530
OOZIE-2528  Print out environment variables in LauncherMapper
https://issues.apache.org/jira/browse/OOZIE-2528
OOZIE-2521  Filter options are case sensitive for jobtye=bundle
https://issues.apache.org/jira/browse/OOZIE-2521
OOZIE-2518  Oozie is creating many files and directories in /tmp per day
https://issues.apache.org/jira/browse/OOZIE-2518
OOZIE-2517  Add support for startCreatedTime and endCreatedTime filters for 
coord and bundles
https://issues.apache.org/jira/browse/OOZIE-2517
OOZIE-2516  Update web service documentation for jobs API
https://issues.apache.org/jira/browse/OOZIE-2516
OOZIE-2513  log.scan.duration should not be used for error and audit logs
https://issues.apache.org/jira/browse/OOZIE-2513
OOZIE-2512  ShareLibservice returns incorrect path for jar
https://issues.apache.org/jira/browse/OOZIE-2512
OOZIE-2508  Documentation change for Coord action rerun [OOZIE-1735]
https://issues.apache.org/jira/browse/OOZIE-2508
OOZIE-2501  ZK reentrant lock doesn't work for few cases
https://issues.apache.org/jira/browse/OOZIE-2501
OOZIE-2500  -DtestJarSimple option mentioned in minioozie doc does not work
https://issues.apache.org/jira/browse/OOZIE-2500
OOZIE-2499  map-reduce launcher does not need distributed files, archives 
except  jar of input/outputformat class
https://issues.apache.org/jira/browse/OOZIE-2499
OOZIE-2498  Oozie CallerId configuration for downstream components 
https://issues.apache.org/jira/browse/OOZIE-2498
OOZIE-2497  Testcase fails on windows with hard coded URIs
https://issues.apache.org/jira/browse/OOZIE-2497
OOZIE-2496  Testcase