[jira] [Updated] (OOZIE-2004) Improve Oozie version info output

[Chunjun Xiao (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-2004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chunjun Xiao updated OOZIE-2004:

Description: 
Current oozie version info output is quite simple:
{code}
[root@svm5 ~] # oozie version
Oozie client build version: 4.0.1
[root@svm5 ~] #
{code}

Can we improve it to follow the convention of  Hadoop/HBase, as:
{code}
[root@svm5 ~] # oozie version
Oozie 4.0.1
Source code repository: ssh://git@XXX/oozie.git -r 
3171cb656a4e4c1b2c1e9c6a47abed7df5bbcc8d
Compiled by root on Sun Jul  6 04:38:08 PDT 2014
>From source with checksum XXX
[root@svm5 ~] #
{code}

  was:
Current oozie version info output is quite simple:
{code}
[root@svm5 ~] # oozie version
Oozie client build version: 4.0.1
[root@svm5 ~] #
{code}

Can we improve it to follow the convention of  Hadoop/HBase, as:
{code}
[root@svm5 ~] # oozie version
Oozie 4.0.1
Source code repository: ssh://git@XXX/oozie.git
git commit id 3171cb656a4e4c1b2c1e9c6a47abed7df5bbcc8d
Compiled by root on Sun Jul  6 04:38:08 PDT 2014
>From source with checksum XXX
[root@svm5 ~] #
{code}


> Improve Oozie version info output
> -
>
> Key: OOZIE-2004
> URL: https://issues.apache.org/jira/browse/OOZIE-2004
> Project: Oozie
>  Issue Type: Improvement
>  Components: build
>Affects Versions: 4.0.1
>Reporter: Chunjun Xiao
>Priority: Minor
>
> Current oozie version info output is quite simple:
> {code}
> [root@svm5 ~] # oozie version
> Oozie client build version: 4.0.1
> [root@svm5 ~] #
> {code}
> Can we improve it to follow the convention of  Hadoop/HBase, as:
> {code}
> [root@svm5 ~] # oozie version
> Oozie 4.0.1
> Source code repository: ssh://git@XXX/oozie.git -r 
> 3171cb656a4e4c1b2c1e9c6a47abed7df5bbcc8d
> Compiled by root on Sun Jul  6 04:38:08 PDT 2014
> From source with checksum XXX
> [root@svm5 ~] #
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-1795) Please create a DOAP file for your TLP

[Sebb (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-1795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14135204#comment-14135204
 ] 

Sebb commented on OOZIE-1795:
-

bq. No. The current setting is fine, and means that the DOAP is stored at
Sorry, my comment above was misleading.

There are actually two RDF files involved.
One for the PMC, and one for each project overseen by the PMC.

The project DOAP points to the PMC RDF, which has already been created for you 
in
{{http://svn.apache.org/repos/asf/infrastructure/site-tools/trunk/projects/data_files/oozie.rdf}}

This can indeed have a list of the PMC members.
If the PMC wishes to maintain this then it may be easier to move it to an Oozie 
repo and change the link to it.
But note that there are other sources of info about the PMC membership:
http://people.apache.org/committers-by-project.html#oozie-pmc
and
https://svn.apache.org/repos/private/committers/board/committee-info.txt (*)
These are also published via whimsy:
https://whimsy.apache.org/roster/committee/oozie (*)

(*) ASF committer login required

> Please create a DOAP file for your TLP
> --
>
> Key: OOZIE-1795
> URL: https://issues.apache.org/jira/browse/OOZIE-1795
> Project: Oozie
>  Issue Type: Task
>Reporter: Sebb
>Assignee: Rohini Palaniswamy
> Attachments: OOZIE-1795-1.patch
>
>
> As per my recent e-mail to your dev list, please can you set up a DOAP for 
> your project and get it added to files.xml?
> Please see http://projects.apache.org/create.html
> Once you have created the DOAP and committed it to your source code 
> repository, please submit it for inclusion in the Apache projects listing as 
> per:
> http://projects.apache.org/create.html#submit
> Remember, if you ever move or rename the doap file in future, please
> ensure that files.xml is updated to point to the new location.
> Thanks!



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2003) Checkstyle issues

[Shwetha G S (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14135296#comment-14135296
 ] 

Shwetha G S commented on OOZIE-2003:


+1

> Checkstyle issues
> -
>
> Key: OOZIE-2003
> URL: https://issues.apache.org/jira/browse/OOZIE-2003
> Project: Oozie
>  Issue Type: Bug
>Reporter: Shwetha G S
>Assignee: Robert Kanter
> Fix For: trunk
>
> Attachments: OOZIE-2003.patch
>
>
> Looks like the build is broken after last commit. [~rkanter], can you fix 
> this please:
> {noformat}
> /work/git/oozie/core/src/main/java/org/apache/oozie/action/hadoop/Hive2ActionExecutor.java:10:
>  Line does not match expected header line of ' *  
> http://www.apache.org/licenses/LICENSE-2.0'.
> /work/git/oozie/core/src/test/java/org/apache/oozie/test/hive/AbstractHiveService.java:10:
>  Line does not match expected header line of ' *  
> http://www.apache.org/licenses/LICENSE-2.0'.
> /work/git/oozie/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java:10:
>  Line does not match expected header line of ' *  
> http://www.apache.org/licenses/LICENSE-2.0'.
> [INFO] Apache Oozie Core . FAILURE [37.299s]
> [ERROR] Failed to execute goal 
> org.apache.maven.plugins:maven-checkstyle-plugin:2.9.1:check (default) on 
> project oozie-core: You have 3 Checkstyle violations. -> [Help 1]
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OOZIE-2003) Checkstyle issues

[Shwetha G S (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-2003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shwetha G S updated OOZIE-2003:
---
Fix Version/s: trunk

> Checkstyle issues
> -
>
> Key: OOZIE-2003
> URL: https://issues.apache.org/jira/browse/OOZIE-2003
> Project: Oozie
>  Issue Type: Bug
>Reporter: Shwetha G S
>Assignee: Robert Kanter
> Fix For: trunk
>
> Attachments: OOZIE-2003.patch
>
>
> Looks like the build is broken after last commit. [~rkanter], can you fix 
> this please:
> {noformat}
> /work/git/oozie/core/src/main/java/org/apache/oozie/action/hadoop/Hive2ActionExecutor.java:10:
>  Line does not match expected header line of ' *  
> http://www.apache.org/licenses/LICENSE-2.0'.
> /work/git/oozie/core/src/test/java/org/apache/oozie/test/hive/AbstractHiveService.java:10:
>  Line does not match expected header line of ' *  
> http://www.apache.org/licenses/LICENSE-2.0'.
> /work/git/oozie/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java:10:
>  Line does not match expected header line of ' *  
> http://www.apache.org/licenses/LICENSE-2.0'.
> [INFO] Apache Oozie Core . FAILURE [37.299s]
> [ERROR] Failed to execute goal 
> org.apache.maven.plugins:maven-checkstyle-plugin:2.9.1:check (default) on 
> project oozie-core: You have 3 Checkstyle violations. -> [Help 1]
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 24165: OOZIE-1954: Add a way for the MapReduce action to be configured by Java code

[shwetha . gs]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24165/#review53508
---

Ship it!


Ship It!

- shwethags


On Sept. 15, 2014, 6:14 p.m., Robert Kanter wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/24165/
> ---
> 
> (Updated Sept. 15, 2014, 6:14 p.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-1954
> https://issues.apache.org/jira/browse/OOZIE-1954
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> See https://issues.apache.org/jira/browse/OOZIE-1954
> 
> 
> Diffs
> -
> 
>   client/src/main/resources/oozie-workflow-0.5.xsd 6620a4e 
>   core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 
> 53f979c 
>   core/src/test/java/org/apache/oozie/action/hadoop/MapperReducerForTest.java 
> d89990c 
>   docs/src/site/twiki/WorkflowFunctionalSpec.twiki 3319bcc 
>   examples/src/main/apps/map-reduce/job-with-config-class.properties 
> PRE-CREATION 
>   examples/src/main/apps/map-reduce/job.properties 7b7a24c 
>   examples/src/main/apps/map-reduce/workflow-with-config-class.xml 
> PRE-CREATION 
>   
> examples/src/main/java/org/apache/oozie/example/SampleOozieActionConfigurator.java
>  PRE-CREATION 
>   
> sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMain.java 
> ed5b88c 
>   
> sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMapper.java
>  ada0706 
>   
> sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/MapReduceMain.java
>  1a9a194 
>   
> sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/OozieActionConfigurator.java
>  PRE-CREATION 
>   
> sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/OozieActionConfiguratorException.java
>  PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PipesMain.java 
> ca32b5f 
>   
> sharelib/streaming/src/test/java/org/apache/oozie/action/hadoop/OozieActionConfiguratorForTest.java
>  PRE-CREATION 
>   
> sharelib/streaming/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java
>  d43ddca 
> 
> Diff: https://reviews.apache.org/r/24165/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Robert Kanter
> 
>

[jira] [Commented] (OOZIE-1954) Add a way for the MapReduce action to be configured by Java code

[Shwetha G S (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-1954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14135305#comment-14135305
 ] 

Shwetha G S commented on OOZIE-1954:


Yes, +1. Thanks

> Add a way for the MapReduce action to be configured by Java code
> 
>
> Key: OOZIE-1954
> URL: https://issues.apache.org/jira/browse/OOZIE-1954
> Project: Oozie
>  Issue Type: New Feature
>Affects Versions: trunk
>Reporter: Robert Kanter
>Assignee: Robert Kanter
> Attachments: OOZIE-1954.patch, OOZIE-1954.patch, OOZIE-1954.patch
>
>
> With certain other components (e.g. Avro, HFileOutputFormat (HBase), etc), it 
> becomes impractical to use the MapReduce action and users must instead use 
> the Java action. The problem is that these components require a lot of extra 
> configuration that is often hidden from the user in Java code (e.g. 
> HFileOutputFormat.configureIncrementalLoad(job, table); which can also 
> include decision logic, serialization, and other things that we can't do in 
> an XML file directly.
> One way to solve this problem is to allow the user to give the MR action some 
> Java code that would do this configuration, similar to how we allow the 
> {{}} field to specify an external XML file of configuration 
> properties.
> In more detail, we could have an interface; something like this:
> {code}
> public interface OozieActionConfigurator {
>  public void updateOozieActionConfiguration(Configuration conf);
> }
> {code}
> that the user can implement, create a jar, and include with their MR action 
> (i.e. add a "{{}}" field that let's them specify the class 
> name). To protect the Oozie server from running user code (which could do 
> anything it wants really), it would have to be run in the Launcher Job. The 
> Launcher Job could call this method after it loads the configuration prepared 
> by the Oozie server.
> Another thing this will be helpful is with users who use the Java action to 
> launch MR jobs and expect a bunch of things to be done for them that are not 
> (e.g. delegation token propagation, config loading, returning the hadoop job 
> to Oozie, etc). These are all done with the MR action, so the more users we 
> can move to the MR action from the Java action, the less they'll run into 
> these difficulties.
> Some of this may change slightly as I try to actually implement this (e.g. 
> have to handle throwing exceptions etc).  And one thing I may do is keep this 
> general enough that it should be compatible with all action types in case we 
> want to add this to any of them in the future; though for now, the schema 
> would only accept it for the MapReduce action.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2003) Checkstyle issues

[Shwetha G S (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14135318#comment-14135318
 ] 

Shwetha G S commented on OOZIE-2003:


Committed to trunk. Thanks [~rkanter]

> Checkstyle issues
> -
>
> Key: OOZIE-2003
> URL: https://issues.apache.org/jira/browse/OOZIE-2003
> Project: Oozie
>  Issue Type: Bug
>Reporter: Shwetha G S
>Assignee: Robert Kanter
> Fix For: trunk
>
> Attachments: OOZIE-2003.patch
>
>
> Looks like the build is broken after last commit. [~rkanter], can you fix 
> this please:
> {noformat}
> /work/git/oozie/core/src/main/java/org/apache/oozie/action/hadoop/Hive2ActionExecutor.java:10:
>  Line does not match expected header line of ' *  
> http://www.apache.org/licenses/LICENSE-2.0'.
> /work/git/oozie/core/src/test/java/org/apache/oozie/test/hive/AbstractHiveService.java:10:
>  Line does not match expected header line of ' *  
> http://www.apache.org/licenses/LICENSE-2.0'.
> /work/git/oozie/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java:10:
>  Line does not match expected header line of ' *  
> http://www.apache.org/licenses/LICENSE-2.0'.
> [INFO] Apache Oozie Core . FAILURE [37.299s]
> [ERROR] Failed to execute goal 
> org.apache.maven.plugins:maven-checkstyle-plugin:2.9.1:check (default) on 
> project oozie-core: You have 3 Checkstyle violations. -> [Help 1]
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-1954) Add a way for the MapReduce action to be configured by Java code

[Shwetha G S (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-1954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14135319#comment-14135319
 ] 

Shwetha G S commented on OOZIE-1954:


Triggered https://builds.apache.org/job/oozie-trunk-precommit-build/1980/

> Add a way for the MapReduce action to be configured by Java code
> 
>
> Key: OOZIE-1954
> URL: https://issues.apache.org/jira/browse/OOZIE-1954
> Project: Oozie
>  Issue Type: New Feature
>Affects Versions: trunk
>Reporter: Robert Kanter
>Assignee: Robert Kanter
> Attachments: OOZIE-1954.patch, OOZIE-1954.patch, OOZIE-1954.patch
>
>
> With certain other components (e.g. Avro, HFileOutputFormat (HBase), etc), it 
> becomes impractical to use the MapReduce action and users must instead use 
> the Java action. The problem is that these components require a lot of extra 
> configuration that is often hidden from the user in Java code (e.g. 
> HFileOutputFormat.configureIncrementalLoad(job, table); which can also 
> include decision logic, serialization, and other things that we can't do in 
> an XML file directly.
> One way to solve this problem is to allow the user to give the MR action some 
> Java code that would do this configuration, similar to how we allow the 
> {{}} field to specify an external XML file of configuration 
> properties.
> In more detail, we could have an interface; something like this:
> {code}
> public interface OozieActionConfigurator {
>  public void updateOozieActionConfiguration(Configuration conf);
> }
> {code}
> that the user can implement, create a jar, and include with their MR action 
> (i.e. add a "{{}}" field that let's them specify the class 
> name). To protect the Oozie server from running user code (which could do 
> anything it wants really), it would have to be run in the Launcher Job. The 
> Launcher Job could call this method after it loads the configuration prepared 
> by the Oozie server.
> Another thing this will be helpful is with users who use the Java action to 
> launch MR jobs and expect a bunch of things to be done for them that are not 
> (e.g. delegation token propagation, config loading, returning the hadoop job 
> to Oozie, etc). These are all done with the MR action, so the more users we 
> can move to the MR action from the Java action, the less they'll run into 
> these difficulties.
> Some of this may change slightly as I try to actually implement this (e.g. 
> have to handle throwing exceptions etc).  And one thing I may do is keep this 
> general enough that it should be compatible with all action types in case we 
> want to add this to any of them in the future; though for now, the schema 
> would only accept it for the MapReduce action.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Build failed in Jenkins: oozie-trunk-precommit-build #1979

[Apache Jenkins Server]

See 

--
[...truncated 16333 lines...]
[INFO] share already added, skipping
[INFO] share/lib already added, skipping
[INFO]  already added, skipping
[INFO] share already added, skipping
[INFO] share/lib already added, skipping
[INFO]  already added, skipping
[INFO] share already added, skipping
[INFO] share/lib already added, skipping
[INFO]  already added, skipping
[INFO] share already added, skipping
[INFO] share/lib already added, skipping
[INFO] 
[INFO] 
[INFO] Building Apache Oozie Tools 4.2.0-SNAPSHOT
[INFO] 
[INFO] 
[INFO] --- maven-resources-plugin:2.5:resources (default-resources) @ 
oozie-tools ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:compile (default-compile) @ oozie-tools 
---
[INFO] Nothing to compile - all classes are up to date
[INFO] 
[INFO] --- maven-resources-plugin:2.5:testResources (default-testResources) @ 
oozie-tools ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 2 resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:testCompile (default-testCompile) @ 
oozie-tools ---
[INFO] Nothing to compile - all classes are up to date
[INFO] 
[INFO] --- maven-surefire-plugin:2.12.2:test (default-test) @ oozie-tools ---
[INFO] Tests are skipped.
[INFO] 
[INFO] --- maven-jar-plugin:2.3.1:jar (default-jar) @ oozie-tools ---
[INFO] Building jar: 

[INFO] 
[INFO] --- maven-assembly-plugin:2.2.1:single (default-cli) @ oozie-tools ---
[INFO] Reading assembly descriptor: ../src/main/assemblies/tools.xml
[WARNING] The following patterns were never triggered in this artifact 
exclusion filter:
o  '*:*:pom:*'

[INFO] Copying files to 

[WARNING] Assembly file: 

 is not a regular file (it may be a directory). It cannot be attached to the 
project build for installation or deployment.
[INFO] 
[INFO] 
[INFO] Building Apache Oozie MiniOozie 4.2.0-SNAPSHOT
[INFO] 
[INFO] 
[INFO] --- maven-resources-plugin:2.5:resources (default-resources) @ 
oozie-mini ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:compile (default-compile) @ oozie-mini 
---
[INFO] No sources to compile
[INFO] 
[INFO] --- maven-resources-plugin:2.5:testResources (default-testResources) @ 
oozie-mini ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:testCompile (default-testCompile) @ 
oozie-mini ---
[INFO] Nothing to compile - all classes are up to date
[INFO] 
[INFO] --- maven-surefire-plugin:2.12.2:test (default-test) @ oozie-mini ---
[INFO] Tests are skipped.
[INFO] 
[INFO] --- maven-jar-plugin:2.3.1:jar (default-jar) @ oozie-mini ---
[WARNING] JAR will be empty - no content was marked for inclusion!
[INFO] Building jar: 

[INFO] 
[INFO] --- maven-assembly-plugin:2.2.1:single (default-cli) @ oozie-mini ---
[INFO] Reading assembly descriptor: src/main/assemblies/empty.xml
[INFO] 
[INFO] 
[INFO] Building Apache Oozie Distro 4.2.0-SNAPSHOT
[INFO] 
[INFO] 
[INFO] --- maven-resources-plugin:2.5:resources (default-resources) @ 
oozie-distro ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:compile (default-compile) @ oozie-distro 
---
[INFO] No

[jira] [Commented] (OOZIE-2003) Checkstyle issues

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14135391#comment-14135391
 ] 

Hadoop QA commented on OOZIE-2003:
--

Testing JIRA OOZIE-2003

Cleaning local git workspace



{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:green}+1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than 
132
.{color:green}+1{color} the patch does adds/modifies 3 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT 
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc 
warnings
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac 
warnings
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA 
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:red}-1 TESTS{color}
.Tests run: 1527
.Tests failed: 1
.Tests errors: 0

.The patch failed the following testcases:

.  
testBundleStatusTransitRunningFromKilled(org.apache.oozie.service.TestStatusTransitService)

{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch 


{color:red}*-1 Overall result, please check the reported -1(s)*{color}


The full output of the test-patch run is available at

.   https://builds.apache.org/job/oozie-trunk-precommit-build/1979/

> Checkstyle issues
> -
>
> Key: OOZIE-2003
> URL: https://issues.apache.org/jira/browse/OOZIE-2003
> Project: Oozie
>  Issue Type: Bug
>Reporter: Shwetha G S
>Assignee: Robert Kanter
> Fix For: trunk
>
> Attachments: OOZIE-2003.patch
>
>
> Looks like the build is broken after last commit. [~rkanter], can you fix 
> this please:
> {noformat}
> /work/git/oozie/core/src/main/java/org/apache/oozie/action/hadoop/Hive2ActionExecutor.java:10:
>  Line does not match expected header line of ' *  
> http://www.apache.org/licenses/LICENSE-2.0'.
> /work/git/oozie/core/src/test/java/org/apache/oozie/test/hive/AbstractHiveService.java:10:
>  Line does not match expected header line of ' *  
> http://www.apache.org/licenses/LICENSE-2.0'.
> /work/git/oozie/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java:10:
>  Line does not match expected header line of ' *  
> http://www.apache.org/licenses/LICENSE-2.0'.
> [INFO] Apache Oozie Core . FAILURE [37.299s]
> [ERROR] Failed to execute goal 
> org.apache.maven.plugins:maven-checkstyle-plugin:2.9.1:check (default) on 
> project oozie-core: You have 3 Checkstyle violations. -> [Help 1]
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-1954) Add a way for the MapReduce action to be configured by Java code

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-1954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14135397#comment-14135397
 ] 

Hadoop QA commented on OOZIE-1954:
--

Testing JIRA OOZIE-1954

Cleaning local git workspace



{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:green}+1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than 
132
.{color:green}+1{color} the patch does adds/modifies 3 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT 
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc 
warnings
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac 
warnings
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA 
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:green}+1 TESTS{color}
.Tests run: 1530
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch 


{color:green}*+1 Overall result, good!, no -1s*{color}


The full output of the test-patch run is available at

.   https://builds.apache.org/job/oozie-trunk-precommit-build/1980/

> Add a way for the MapReduce action to be configured by Java code
> 
>
> Key: OOZIE-1954
> URL: https://issues.apache.org/jira/browse/OOZIE-1954
> Project: Oozie
>  Issue Type: New Feature
>Affects Versions: trunk
>Reporter: Robert Kanter
>Assignee: Robert Kanter
> Attachments: OOZIE-1954.patch, OOZIE-1954.patch, OOZIE-1954.patch
>
>
> With certain other components (e.g. Avro, HFileOutputFormat (HBase), etc), it 
> becomes impractical to use the MapReduce action and users must instead use 
> the Java action. The problem is that these components require a lot of extra 
> configuration that is often hidden from the user in Java code (e.g. 
> HFileOutputFormat.configureIncrementalLoad(job, table); which can also 
> include decision logic, serialization, and other things that we can't do in 
> an XML file directly.
> One way to solve this problem is to allow the user to give the MR action some 
> Java code that would do this configuration, similar to how we allow the 
> {{}} field to specify an external XML file of configuration 
> properties.
> In more detail, we could have an interface; something like this:
> {code}
> public interface OozieActionConfigurator {
>  public void updateOozieActionConfiguration(Configuration conf);
> }
> {code}
> that the user can implement, create a jar, and include with their MR action 
> (i.e. add a "{{}}" field that let's them specify the class 
> name). To protect the Oozie server from running user code (which could do 
> anything it wants really), it would have to be run in the Launcher Job. The 
> Launcher Job could call this method after it loads the configuration prepared 
> by the Oozie server.
> Another thing this will be helpful is with users who use the Java action to 
> launch MR jobs and expect a bunch of things to be done for them that are not 
> (e.g. delegation token propagation, config loading, returning the hadoop job 
> to Oozie, etc). These are all done with the MR action, so the more users we 
> can move to the MR action from the Java action, the less they'll run into 
> these difficulties.
> Some of this may change slightly as I try to actually implement this (e.g. 
> have to handle throwing exceptions etc).  And one thing I may do is keep this 
> general enough that it should be compatible with all action types in case we 
> want to add this to any of them in the future; though for now, the schema 
> would only accept it for the MapReduce action.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Jenkins build is back to normal : oozie-trunk-precommit-build #1980

[Apache Jenkins Server]

See 

[jira] [Commented] (OOZIE-1795) Please create a DOAP file for your TLP

[Rohini Palaniswamy (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-1795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14135684#comment-14135684
 ] 

Rohini Palaniswamy commented on OOZIE-1795:
---

[~s...@apache.org],
Thanks for the detailed info. Will change http://oozie.apache.org"; /> to http://people.apache.org/committers-by-project.html#oozie-pmc"; /> 
so that we don't have to maintain another RDF file and remember to update it. 
That means we will also have to remove the entry in 
http://svn.apache.org/repos/asf/infrastructure/site-tools/trunk/projects/pmc_list.xml
 right?

> Please create a DOAP file for your TLP
> --
>
> Key: OOZIE-1795
> URL: https://issues.apache.org/jira/browse/OOZIE-1795
> Project: Oozie
>  Issue Type: Task
>Reporter: Sebb
>Assignee: Rohini Palaniswamy
> Attachments: OOZIE-1795-1.patch
>
>
> As per my recent e-mail to your dev list, please can you set up a DOAP for 
> your project and get it added to files.xml?
> Please see http://projects.apache.org/create.html
> Once you have created the DOAP and committed it to your source code 
> repository, please submit it for inclusion in the Apache projects listing as 
> per:
> http://projects.apache.org/create.html#submit
> Remember, if you ever move or rename the doap file in future, please
> ensure that files.xml is updated to point to the new location.
> Thanks!



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-1795) Please create a DOAP file for your TLP

[Sebb (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-1795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14135697#comment-14135697
 ] 

Sebb commented on OOZIE-1795:
-

*Please don't. That won't work.*

rdf:resource has to point to an RDF (XML) file; the people URL is an 
unstructured HTML file.

The people URL is automatically maintained from LDAP, so you may consider it 
unnecessary to maintain a separate list in the oozie.rdf file. But *if you do 
wish to do so*, it would probably be a good idea to move the file to a repo 
which is writeable by all the Oozie committers (or at least all the PMC).

> Please create a DOAP file for your TLP
> --
>
> Key: OOZIE-1795
> URL: https://issues.apache.org/jira/browse/OOZIE-1795
> Project: Oozie
>  Issue Type: Task
>Reporter: Sebb
>Assignee: Rohini Palaniswamy
> Attachments: OOZIE-1795-1.patch
>
>
> As per my recent e-mail to your dev list, please can you set up a DOAP for 
> your project and get it added to files.xml?
> Please see http://projects.apache.org/create.html
> Once you have created the DOAP and committed it to your source code 
> repository, please submit it for inclusion in the Apache projects listing as 
> per:
> http://projects.apache.org/create.html#submit
> Remember, if you ever move or rename the doap file in future, please
> ensure that files.xml is updated to point to the new location.
> Thanks!



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-1795) Please create a DOAP file for your TLP

[Sebb (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-1795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14135710#comment-14135710
 ] 

Sebb commented on OOZIE-1795:
-

bq. The change could be found at : 
http://svn.apache.org/repos/asf/infrastructure/site-tools/trunk/projects/pmc_list.xml

Wrong file name in the comment. The file that was updated was actually:

https://svn.apache.org/repos/asf/infrastructure/site-tools/trunk/projects/files.xml

This is the correct file for DOAPs.

The file shown in the comment is for PMC descriptors.

> Please create a DOAP file for your TLP
> --
>
> Key: OOZIE-1795
> URL: https://issues.apache.org/jira/browse/OOZIE-1795
> Project: Oozie
>  Issue Type: Task
>Reporter: Sebb
>Assignee: Rohini Palaniswamy
> Attachments: OOZIE-1795-1.patch
>
>
> As per my recent e-mail to your dev list, please can you set up a DOAP for 
> your project and get it added to files.xml?
> Please see http://projects.apache.org/create.html
> Once you have created the DOAP and committed it to your source code 
> repository, please submit it for inclusion in the Apache projects listing as 
> per:
> http://projects.apache.org/create.html#submit
> Remember, if you ever move or rename the doap file in future, please
> ensure that files.xml is updated to point to the new location.
> Thanks!



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-1795) Please create a DOAP file for your TLP

[Sebb (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-1795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14135715#comment-14135715
 ] 

Sebb commented on OOZIE-1795:
-

bq. That means we will also have to remove the entry in 
http://svn.apache.org/repos/asf/infrastructure/site-tools/trunk/projects/pmc_list.xml
 right?

No, every PMC must have an entry in that file.
Those that wish to maintain their own PMC RDF file need to ensure the URL is 
updated to point to the correct location.

> Please create a DOAP file for your TLP
> --
>
> Key: OOZIE-1795
> URL: https://issues.apache.org/jira/browse/OOZIE-1795
> Project: Oozie
>  Issue Type: Task
>Reporter: Sebb
>Assignee: Rohini Palaniswamy
> Attachments: OOZIE-1795-1.patch
>
>
> As per my recent e-mail to your dev list, please can you set up a DOAP for 
> your project and get it added to files.xml?
> Please see http://projects.apache.org/create.html
> Once you have created the DOAP and committed it to your source code 
> repository, please submit it for inclusion in the Apache projects listing as 
> per:
> http://projects.apache.org/create.html#submit
> Remember, if you ever move or rename the doap file in future, please
> ensure that files.xml is updated to point to the new location.
> Thanks!



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-1795) Please create a DOAP file for your TLP

[Rohini Palaniswamy (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-1795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14135738#comment-14135738
 ] 

Rohini Palaniswamy commented on OOZIE-1795:
---

[~s...@apache.org],
Sorry. Got it. Thanks for being patient and clarifying. It should be of the 
format described in http://projects.apache.org/docs/pmc.html. I was checking 
out few files in 
http://svn.apache.org/repos/asf/infrastructure/site-tools/trunk/projects/data_files/
 . Most of them are empty. Few have just the chair and only axis.rdf listed all 
the members. So I am assuming most projects chose not to maintain another list 
separate from the people URL. We will also go with that.  Curiosity question - 
Why do we need another doap rdf file for PMC when it can point to the people 
URL. Most of the other entries in project doap file take a http URL pointing to 
a HTML file. Can't we just make PMC also take one?

> Please create a DOAP file for your TLP
> --
>
> Key: OOZIE-1795
> URL: https://issues.apache.org/jira/browse/OOZIE-1795
> Project: Oozie
>  Issue Type: Task
>Reporter: Sebb
>Assignee: Rohini Palaniswamy
> Attachments: OOZIE-1795-1.patch
>
>
> As per my recent e-mail to your dev list, please can you set up a DOAP for 
> your project and get it added to files.xml?
> Please see http://projects.apache.org/create.html
> Once you have created the DOAP and committed it to your source code 
> repository, please submit it for inclusion in the Apache projects listing as 
> per:
> http://projects.apache.org/create.html#submit
> Remember, if you ever move or rename the doap file in future, please
> ensure that files.xml is updated to point to the new location.
> Thanks!



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-1795) Please create a DOAP file for your TLP

[Sebb (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-1795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14135756#comment-14135756
 ] 

Sebb commented on OOZIE-1795:
-

bq. Most of them are empty.

None are empty, but many only have the minimum contents needed for the website 
to be created

bq. Why do we need another doap rdf file for PMC when it can point to the 
people URL

It requires an RDF (XML) file.
As already explained, the people URL is not suitable.

bq. Can't we just make PMC also take one?

No, the information has to be in RDF (XML) format.

> Please create a DOAP file for your TLP
> --
>
> Key: OOZIE-1795
> URL: https://issues.apache.org/jira/browse/OOZIE-1795
> Project: Oozie
>  Issue Type: Task
>Reporter: Sebb
>Assignee: Rohini Palaniswamy
> Attachments: OOZIE-1795-1.patch
>
>
> As per my recent e-mail to your dev list, please can you set up a DOAP for 
> your project and get it added to files.xml?
> Please see http://projects.apache.org/create.html
> Once you have created the DOAP and committed it to your source code 
> repository, please submit it for inclusion in the Apache projects listing as 
> per:
> http://projects.apache.org/create.html#submit
> Remember, if you ever move or rename the doap file in future, please
> ensure that files.xml is updated to point to the new location.
> Thanks!



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (OOZIE-1795) Please create a DOAP file for your TLP

[Rohini Palaniswamy (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-1795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rohini Palaniswamy resolved OOZIE-1795.
---
Resolution: Fixed

bq. None are empty, but many only have the minimum contents needed for the 
website to be created
  That's right. I actually meant that they don't have any PMC information when 
I said they are empty.

Thanks [~s...@apache.org]. 

Required work is done, but will resolve the jira fixed once Oozie shows up in 
http://projects.apache.org/indexes/alpha.html#O

> Please create a DOAP file for your TLP
> --
>
> Key: OOZIE-1795
> URL: https://issues.apache.org/jira/browse/OOZIE-1795
> Project: Oozie
>  Issue Type: Task
>Reporter: Sebb
>Assignee: Rohini Palaniswamy
> Attachments: OOZIE-1795-1.patch
>
>
> As per my recent e-mail to your dev list, please can you set up a DOAP for 
> your project and get it added to files.xml?
> Please see http://projects.apache.org/create.html
> Once you have created the DOAP and committed it to your source code 
> repository, please submit it for inclusion in the Apache projects listing as 
> per:
> http://projects.apache.org/create.html#submit
> Remember, if you ever move or rename the doap file in future, please
> ensure that files.xml is updated to point to the new location.
> Thanks!



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-1917) Authentication secret should be random by default and needs to coordinate with HA

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-1917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14136030#comment-14136030
 ] 

Hadoop QA commented on OOZIE-1917:
--

Testing JIRA OOZIE-1917

Cleaning local git workspace



{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:red}-1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than 
132
.{color:red}-1{color} the patch does not add/modify any testcase
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT 
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc 
warnings
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac 
warnings
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA 
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:red}-1 TESTS{color}
.Tests run: 1527
.Tests failed: 2
.Tests errors: 0

.The patch failed the following testcases:

.  
testMessage_withMixedStatus(org.apache.oozie.command.coord.TestAbandonedCoordChecker)
.  
testBundleRerunInPausedWithError(org.apache.oozie.command.bundle.TestBundleRerunXCommand)

{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch 


{color:red}*-1 Overall result, please check the reported -1(s)*{color}


The full output of the test-patch run is available at

.   https://builds.apache.org/job/oozie-trunk-precommit-build/1981/

> Authentication secret should be random by default and needs to coordinate 
> with HA
> -
>
> Key: OOZIE-1917
> URL: https://issues.apache.org/jira/browse/OOZIE-1917
> Project: Oozie
>  Issue Type: Improvement
>  Components: HA, security
>Affects Versions: trunk
>Reporter: Robert Kanter
>Assignee: Robert Kanter
>Priority: Critical
> Attachments: OOZIE-1917.patch
>
>
> {{oozie.authentication.signature.secret}} is currently set to {{oozie}} by 
> default, which is a pretty poor value for this.  We should set it to be 
> random by default (i.e. blank in oozie-site/default).  
> We should also make it so that with Oozie HA, we store this value in 
> ZooKeeper so all Oozie servers can use the same secret.  This may get a 
> little tricky because hadoop-auth's AuthenticationFilter doesn't make it 
> easy/practical to change how the Signer and secret are set.  We'll likely 
> have to have Oozie's AuthFilter compute it's own random secret and do all the 
> ZK stuff and set the value of {{oozie.authentication.signature.secret}} 
> before calling AuthenticationFilter#init



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Build failed in Jenkins: oozie-trunk-precommit-build #1981

[Apache Jenkins Server]

See 

--
[...truncated 10121 lines...]
[INFO] share/lib already added, skipping
[INFO]  already added, skipping
[INFO] share already added, skipping
[INFO] share/lib already added, skipping
[INFO]  already added, skipping
[INFO] share already added, skipping
[INFO] share/lib already added, skipping
[INFO]  already added, skipping
[INFO] share already added, skipping
[INFO] share/lib already added, skipping
[INFO] 
[INFO] 
[INFO] Building Apache Oozie Tools 4.2.0-SNAPSHOT
[INFO] 
[INFO] 
[INFO] --- maven-resources-plugin:2.5:resources (default-resources) @ 
oozie-tools ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:compile (default-compile) @ oozie-tools 
---
[INFO] Nothing to compile - all classes are up to date
[INFO] 
[INFO] --- maven-resources-plugin:2.5:testResources (default-testResources) @ 
oozie-tools ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 2 resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:testCompile (default-testCompile) @ 
oozie-tools ---
[INFO] Nothing to compile - all classes are up to date
[INFO] 
[INFO] --- maven-surefire-plugin:2.12.2:test (default-test) @ oozie-tools ---
[INFO] Tests are skipped.
[INFO] 
[INFO] --- maven-jar-plugin:2.3.1:jar (default-jar) @ oozie-tools ---
[INFO] Building jar: 

[INFO] 
[INFO] --- maven-assembly-plugin:2.2.1:single (default-cli) @ oozie-tools ---
[INFO] Reading assembly descriptor: ../src/main/assemblies/tools.xml
[WARNING] The following patterns were never triggered in this artifact 
exclusion filter:
o  '*:*:pom:*'

[INFO] Copying files to 

[WARNING] Assembly file: 

 is not a regular file (it may be a directory). It cannot be attached to the 
project build for installation or deployment.
[INFO] 
[INFO] 
[INFO] Building Apache Oozie MiniOozie 4.2.0-SNAPSHOT
[INFO] 
[INFO] 
[INFO] --- maven-resources-plugin:2.5:resources (default-resources) @ 
oozie-mini ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:compile (default-compile) @ oozie-mini 
---
[INFO] No sources to compile
[INFO] 
[INFO] --- maven-resources-plugin:2.5:testResources (default-testResources) @ 
oozie-mini ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:testCompile (default-testCompile) @ 
oozie-mini ---
[INFO] Nothing to compile - all classes are up to date
[INFO] 
[INFO] --- maven-surefire-plugin:2.12.2:test (default-test) @ oozie-mini ---
[INFO] Tests are skipped.
[INFO] 
[INFO] --- maven-jar-plugin:2.3.1:jar (default-jar) @ oozie-mini ---
[WARNING] JAR will be empty - no content was marked for inclusion!
[INFO] Building jar: 

[INFO] 
[INFO] --- maven-assembly-plugin:2.2.1:single (default-cli) @ oozie-mini ---
[INFO] Reading assembly descriptor: src/main/assemblies/empty.xml
[INFO] 
[INFO] 
[INFO] Building Apache Oozie Distro 4.2.0-SNAPSHOT
[INFO] 
[INFO] 
[INFO] --- maven-resources-plugin:2.5:resources (default-resources) @ 
oozie-distro ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:compile (default-compile) @ oozie-distro 
---
[INFO] No sources to compile
[INFO] 
[INFO] --

[jira] [Created] (OOZIE-2005) Coordinator rerun fails to initialize error code and message.

[Ryota Egashira (JIRA)]

Ryota Egashira created OOZIE-2005:
-

 Summary: Coordinator rerun fails to initialize error code and 
message.
 Key: OOZIE-2005
 URL: https://issues.apache.org/jira/browse/OOZIE-2005
 Project: Oozie
  Issue Type: Bug
Reporter: Ryota Egashira
Assignee: Ryota Egashira


Error code and message of previous run remain on UI even when rerun succeeds



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Review Request 25708: OOZIE-2005 Coordinator rerun fails to initialize error code and message

[Ryota Egashira]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25708/
---

Review request for oozie.


Repository: oozie-git


Description
---

https://issues.apache.org/jira/browse/OOZIE-2005


Diffs
-

  core/src/main/java/org/apache/oozie/CoordinatorActionBean.java cc5596b 
  core/src/main/java/org/apache/oozie/command/coord/CoordRerunXCommand.java 
e70343b 
  
core/src/main/java/org/apache/oozie/executor/jpa/CoordActionQueryExecutor.java 
0aee0e4 
  core/src/main/java/org/apache/oozie/store/CoordinatorStore.java aeb7d38 
  core/src/test/java/org/apache/oozie/command/coord/TestCoordRerunXCommand.java 
378042c 

Diff: https://reviews.apache.org/r/25708/diff/


Testing
---


Thanks,

Ryota Egashira

[jira] [Commented] (OOZIE-1917) Authentication secret should be random by default and needs to coordinate with HA

[Robert Kanter (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-1917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14136112#comment-14136112
 ] 

Robert Kanter commented on OOZIE-1917:
--

Test failures are unrelated.  No tests because this would be difficult to test 
and hadoop-auth has some related tests on this already.  I did try it out in an 
actual cluster though.

> Authentication secret should be random by default and needs to coordinate 
> with HA
> -
>
> Key: OOZIE-1917
> URL: https://issues.apache.org/jira/browse/OOZIE-1917
> Project: Oozie
>  Issue Type: Improvement
>  Components: HA, security
>Affects Versions: trunk
>Reporter: Robert Kanter
>Assignee: Robert Kanter
>Priority: Critical
> Attachments: OOZIE-1917.patch
>
>
> {{oozie.authentication.signature.secret}} is currently set to {{oozie}} by 
> default, which is a pretty poor value for this.  We should set it to be 
> random by default (i.e. blank in oozie-site/default).  
> We should also make it so that with Oozie HA, we store this value in 
> ZooKeeper so all Oozie servers can use the same secret.  This may get a 
> little tricky because hadoop-auth's AuthenticationFilter doesn't make it 
> easy/practical to change how the Signer and secret are set.  We'll likely 
> have to have Oozie's AuthFilter compute it's own random secret and do all the 
> ZK stuff and set the value of {{oozie.authentication.signature.secret}} 
> before calling AuthenticationFilter#init



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-1876) use pom properties rather than specific version numbers in the pom files of hbaselibs, hcataloglibs, sharelib, etc

[Robert Kanter (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-1876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14136168#comment-14136168
 ] 

Robert Kanter commented on OOZIE-1876:
--

Looks good to me.  A few minor things:
# It doesn't work now because of the new "hive2" sharelib added by OOZIE-1457; 
it's still using "oozie-hcatalog":
{noformat}
[INFO] Scanning for projects...
[ERROR] The build could not read 1 project -> [Help 1]
[ERROR]   
[ERROR]   The project org.apache.oozie:oozie-sharelib-hive2:4.2.0-SNAPSHOT 
(/Users/rkanter/dev/oozie-git/sharelib/hive2/pom.xml) has 1 error
[ERROR] 'dependencies.dependency.version' for 
org.apache.oozie:oozie-hcatalog:jar is missing. @ line 423, column 21
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e 
switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please 
read the following articles:
[ERROR] [Help 1] 
http://cwiki.apache.org/confluence/display/MAVEN/ProjectBuildingException
{noformat}
# I think adding the documentation about these versions to the QuickStart is a 
good idea.  But it's currently in a section talking about skipping tests.  I 
think this should be in it's own section, and we should explain a bit more 
about it.  For example, it would be good to warn users that (a) Oozie isn't 
tested against every version of each component so it might not work correctly 
even if it compiles and (b) it might not always compile.  

+1 after that

> use pom properties rather than specific version numbers in the pom files of 
> hbaselibs, hcataloglibs, sharelib, etc 
> ---
>
> Key: OOZIE-1876
> URL: https://issues.apache.org/jira/browse/OOZIE-1876
> Project: Oozie
>  Issue Type: Improvement
>  Components: build
>Affects Versions: 4.0.1
>Reporter: John
>Assignee: Shwetha G S
> Fix For: trunk
>
> Attachments: OOZIE-1876-v2.patch, OOZIE-1876-v3.patch, 
> OOZIE-1876-v4.patch, OOZIE-1876-v5.patch, OOZIE-1876-v5.patch, 
> OOZIE-1876.patch
>
>
> version numbers (hbase, hive, hcatalog, sqoop, etc) are hard coded in the pom 
> files.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (OOZIE-2006) Hive 2 action should support tez delegation tokens

[Robert Kanter (JIRA)]

Robert Kanter created OOZIE-2006:


 Summary: Hive 2 action should support tez delegation tokens
 Key: OOZIE-2006
 URL: https://issues.apache.org/jira/browse/OOZIE-2006
 Project: Oozie
  Issue Type: Sub-task
  Components: action
Affects Versions: trunk
Reporter: Robert Kanter
Assignee: Bowen Zhang


Hive2Main needs to propagate the delegation token for tez; like HiveMain :
{code:java}
hiveConf.set("tez.credentials.path", delegationToken);
{code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (OOZIE-2007) no input data for hive example workflow

[Gordon Wang (JIRA)]

Gordon Wang created OOZIE-2007:
--

 Summary: no input data for hive example workflow
 Key: OOZIE-2007
 URL: https://issues.apache.org/jira/browse/OOZIE-2007
 Project: Oozie
  Issue Type: Bug
  Components: examples
Reporter: Gordon Wang


In hive examples, the input data should be in 
"/user/${wf:user()}/${examplesRoot}/input-data/table". But currently, no data 
is in input-data/table. User have to prepare data by themselves. 

I need to put some example data in input-data/table. And make sure user can run 
the example smoothly.
The following aspects should be changed.
# put "DROP TABLE IF EXISTS test;" in script.q. Make sure the table does not 
exist.
# put some test data in input-data/table.
# add "${nameNode}/user/oozie/hive-oozie-site.xml" in 
workflow.xml. Then, user can use the hive conf in their cluster.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OOZIE-2008) When oozie war is not setup, oozie service fails to start

[Gordon Wang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-2008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gordon Wang updated OOZIE-2008:
---
Summary: When oozie war is not setup, oozie service fails to start  (was: 
When oozie war is not setup, oozie service start fails)

> When oozie war is not setup, oozie service fails to start
> -
>
> Key: OOZIE-2008
> URL: https://issues.apache.org/jira/browse/OOZIE-2008
> Project: Oozie
>  Issue Type: Bug
>Reporter: Gordon Wang
>
> When I start Oozie service without setting up Oozie war file, the oozied.sh 
> fails.
> Check the source code further. I think it is a bug in oozied.sh. When oozie 
> war is not setup, oozied.sh tries to prepare the war file. See
> {code}
> setup_oozie() {
>   if [ ! -e "${CATALINA_BASE}/webapps/oozie.war" ]; then
> echo "WARN: Oozie WAR has not been set up at 
> ''${CATALINA_BASE}/webapps'', d
> oing default set up"
> ${BASEDIR}/bin/oozie-setup.sh
> if [ "$?" != "0" ]; then
>   exit -1
> fi
>   fi
>   echo
> }
> {code}
> But we should call {noformat}"${BASEDIR}/bin/oozie-setup.sh 
> prepare-war"{noformat} to prepare the war file with default configuration.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (OOZIE-2008) When oozie war is not setup, oozie service start fails

[Gordon Wang (JIRA)]

Gordon Wang created OOZIE-2008:
--

 Summary: When oozie war is not setup, oozie service start fails
 Key: OOZIE-2008
 URL: https://issues.apache.org/jira/browse/OOZIE-2008
 Project: Oozie
  Issue Type: Bug
Reporter: Gordon Wang


When I start Oozie service without setting up Oozie war file, the oozied.sh 
fails.

Check the source code further. I think it is a bug in oozied.sh. When oozie war 
is not setup, oozied.sh tries to prepare the war file. See
{code}
setup_oozie() {
  if [ ! -e "${CATALINA_BASE}/webapps/oozie.war" ]; then
echo "WARN: Oozie WAR has not been set up at ''${CATALINA_BASE}/webapps'', d
oing default set up"
${BASEDIR}/bin/oozie-setup.sh
if [ "$?" != "0" ]; then
  exit -1
fi
  fi
  echo
}
{code}

But we should call {noformat}"${BASEDIR}/bin/oozie-setup.sh 
prepare-war"{noformat} to prepare the war file with default configuration.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OOZIE-1391) Sub wf suspend doesn't update parent wf

[Jaydeep Vishwakarma (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-1391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jaydeep Vishwakarma updated OOZIE-1391:
---
Attachment: suspend-1.patch

Updating parent status as well.

> Sub wf suspend doesn't update parent wf
> ---
>
> Key: OOZIE-1391
> URL: https://issues.apache.org/jira/browse/OOZIE-1391
> Project: Oozie
>  Issue Type: Bug
>Reporter: Shwetha G S
>Assignee: Jaydeep Vishwakarma
> Attachments: suspend-1.patch, suspend.patch
>
>
> If a workflow contains sub workflow, and sub workflow gets suspended, 
> workflow stays in running state. The workflow should also move to suspended 
> state



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)