Re: Including *-site.xml files in Oozie actions

[Rohini Palaniswamy]

bq. Would it make sense to have this logic in the action executors
themselves, so that the configuration is always loaded during the action
execution?
   Where would the configuration be located when you are trying to load in
the action executors? I am not sure if you can get around having the file
in a common location and updating that file (hive-site.xml, hbase-site.xml,
sqoop-site.xml) when settings change.

Regards,
Rohini

On Thu, Oct 23, 2014 at 12:52 PM, David Villegas  wrote:

> Hi all,
>
> We have been trying to find a sane way of running Oozie workflows that load
> other components' *-site.xml files, and it seems there is no standard way
> of doing this?
>
> To give an example, we wanted to run a Sqoop action that uses the plugins
> defined in sqoop-site.xml. The most common solution I see is copying
> sqoop-site.xml to the sharelib (putting it somewhere else than the sqoop
> root, since it could conflict with the sqoop-site.xml file created by the
> Sqoop action), and then specifying its location in the  element. A
> similar approach needs to be taken for Hive. One of the downsides is the
> configuration file needs to be updated in HDFS every time there's a change
> in the original one. The other approach is bundling the xml file in the
> workflow, but this needs to be done every time a workflow is deployed.
>
> Now, it seems to me loading a *-site.xml file for an action should be a
> common case. Would it make sense to have this logic in the action executors
> themselves, so that the configuration is always loaded during the action
> execution?
>
> Let me know if I am missing something, I'd be ok working on a solution if
> this makes sense.
>
> Regards,
> David Villegas.
>

Re: Review Request 27118: OOZIE-2045 Symlink support for sharelib

[Rohini Palaniswamy]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27118/#review59054
---



core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java
<https://reviews.apache.org/r/27118/#comment100331>

Call it HadoopShims.class. Just for the sake of sticking to the same naming 
convention used by other projects (pig, hive, hbase, etc) when they deal with 
multiple versions of hadoop. Easy to look for by folks familiar with other 
projects.



core/src/main/java/org/apache/oozie/service/ShareLibService.java
<https://reviews.apache.org/r/27118/#comment100332>

unused import



core/src/main/java/org/apache/oozie/service/ShareLibService.java
<https://reviews.apache.org/r/27118/#comment100356>

Add a comment saying

// Map>



core/src/main/java/org/apache/oozie/service/ShareLibService.java
<https://reviews.apache.org/r/27118/#comment100363>

If it is a file, it is not being added to sharelib now.



core/src/main/java/org/apache/oozie/service/ShareLibService.java
<https://reviews.apache.org/r/27118/#comment100347>

Why create so many instances of new HadoopFileSystem(fs). Can instantiate 
once after fs instantiation.

Just log if the file is a symlink with the information that it is a 
symlink. Currently only path space true/false is printed. Also you can just log 
once for the rootDir instead of every file.



core/src/main/java/org/apache/oozie/service/ShareLibService.java
<https://reviews.apache.org/r/27118/#comment100350>

private



core/src/main/java/org/apache/oozie/service/ShareLibService.java
<https://reviews.apache.org/r/27118/#comment100358>

This comment is wrong. Should be for getSystemLibJars method



core/src/main/java/org/apache/oozie/service/ShareLibService.java
<https://reviews.apache.org/r/27118/#comment100360>

sharelibName instead of actionKey



core/src/main/java/org/apache/oozie/service/ShareLibService.java
<https://reviews.apache.org/r/27118/#comment100361>

sharelibName instead of actionKey



core/src/main/java/org/apache/oozie/service/ShareLibService.java
<https://reviews.apache.org/r/27118/#comment100351>

Why would fs be null?



core/src/main/java/org/apache/oozie/service/ShareLibService.java
<https://reviews.apache.org/r/27118/#comment100352>

Symlink target of [{0}] is changed



core/src/main/java/org/apache/oozie/service/ShareLibService.java
<https://reviews.apache.org/r/27118/#comment100364>

You should just reload for that particular sharelib name instead of 
reloading the whole sharelib or meta file again. That will involve keeping the 
contents of the metafile also in memory. Do not want to reload the metafile 
unless a sharelib update command is issued.



core/src/main/java/org/apache/oozie/service/ShareLibService.java
<https://reviews.apache.org/r/27118/#comment100365>

shareLibSymlinkMapping



core/src/test/java/org/apache/oozie/service/TestShareLibService.java
<https://reviews.apache.org/r/27118/#comment100368>

%s/sysmlink/symlink/g



core/src/test/java/org/apache/oozie/service/TestShareLibService.java
<https://reviews.apache.org/r/27118/#comment100369>

new HadoopFileSystem(fs) - instantiate once



core/src/test/java/org/apache/oozie/service/TestShareLibService.java
<https://reviews.apache.org/r/27118/#comment100372>

createTestShareLibMetaFile



hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/hadoop/utils/HadoopFileSystem.java
<https://reviews.apache.org/r/27118/#comment100354>

getSymLinkTarget()



hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/hadoop/utils/HadoopFileSystem.java
<https://reviews.apache.org/r/27118/#comment100355>

annotate with @VisibleForTesting in all the classes


- Rohini Palaniswamy


On Oct. 24, 2014, 5:45 p.m., Purshotam Shah wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/27118/
> ---
> 
> (Updated Oct. 24, 2014, 5:45 p.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-2045
> https://issues.apache.org/jira/browse/OOZIE-2045
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> OOZIE-2045 Symlink support for sharelib
> 
> 
> Diffs
> -
> 
>   core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 
> 201cfa3 
>   core/src/main/java/org/apache/oozie/service/ShareLibService.java 63c5a41 
>   core/src/main/resources/oozie-default.xml 26eb7e0 
>   core/src/test/java/org/apache/oozie/service/TestShareLibService.java 
> 68cb357 
>   
> hadooplibs/had

Re: Review Request 27303: OOZIE-2035 NotificationXCommand should support proxy

[Rohini Palaniswamy]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27303/#review59584
---


Once OOZIE-1890 goes in this patch will have to be rebased for 
Services.getConf()


core/src/main/java/org/apache/oozie/command/NotificationXCommand.java
<https://reviews.apache.org/r/27303/#comment100864>

!proxyConf.trim().equals("")



core/src/main/java/org/apache/oozie/command/NotificationXCommand.java
<https://reviews.apache.org/r/27303/#comment100863>

Remove WorkflowNotificationXCommand.



core/src/main/java/org/apache/oozie/command/coord/CoordActionNotificationXCommand.java
<https://reviews.apache.org/r/27303/#comment100861>

This is missing in new loadState()



docs/src/site/twiki/CoordinatorFunctionalSpec.twiki
<https://reviews.apache.org/r/27303/#comment100865>

=oozie.coord.action.notification.proxy= property can be used to configure 
either a http or socks proxy. The format is proxyHostname:port or 
proxyType@proxyHostname:port. If proxy type is not specified, it defaults to 
http. For eg: myhttpproxyhost.mydomain.com:80 or 
so...@mysockshost.mydomain.com:1080.

Same to be copied for workflow.


- Rohini Palaniswamy


On Oct. 28, 2014, 8:46 p.m., Purshotam Shah wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/27303/
> ---
> 
> (Updated Oct. 28, 2014, 8:46 p.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-2035
> https://issues.apache.org/jira/browse/OOZIE-2035
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> NotificationXCommand should support proxy
> 
> 
> Diffs
> -
> 
>   client/src/main/java/org/apache/oozie/client/OozieClient.java 5e53a18 
>   core/src/main/java/org/apache/oozie/command/NotificationXCommand.java 
> e69de29 
>   
> core/src/main/java/org/apache/oozie/command/coord/CoordActionNotificationXCommand.java
>  d220434 
>   core/src/main/java/org/apache/oozie/command/wf/ActionKillXCommand.java 
> 6073e4d 
>   core/src/main/java/org/apache/oozie/command/wf/ActionStartXCommand.java 
> 373c942 
>   core/src/main/java/org/apache/oozie/command/wf/ActionXCommand.java 98fddfd 
>   core/src/main/java/org/apache/oozie/command/wf/KillXCommand.java cac1381 
>   core/src/main/java/org/apache/oozie/command/wf/ResumeXCommand.java 73c1447 
>   core/src/main/java/org/apache/oozie/command/wf/SignalXCommand.java 5ff0e5c 
>   core/src/main/java/org/apache/oozie/command/wf/SuspendXCommand.java 199af36 
>   
> core/src/main/java/org/apache/oozie/command/wf/WorkflowNotificationXCommand.java
>  73ce9a7 
>   core/src/main/resources/oozie-default.xml 26eb7e0 
>   
> core/src/test/java/org/apache/oozie/command/coord/TestCoordActionNotificationXCommand.java
>  7742dd4 
>   
> core/src/test/java/org/apache/oozie/command/wf/TestWorkflowNotificationXCommand.java
>  b260747 
>   docs/src/site/twiki/CoordinatorFunctionalSpec.twiki adf4ae1 
>   docs/src/site/twiki/WorkflowFunctionalSpec.twiki e6e52a0 
> 
> Diff: https://reviews.apache.org/r/27303/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Purshotam Shah
> 
>

Re: Review Request 25269: OOZIE-1985 support dropping hcat dataset in coord rerun with cleanup option

[Rohini Palaniswamy]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25269/#review59601
---



core/src/main/java/org/apache/oozie/ErrorCode.java
<https://reviews.apache.org/r/25269/#comment100895>

One code for "Failed to delete uri : {0}" should be good enough. User can 
make out hdfs or hcat from the uri.



core/src/main/java/org/apache/oozie/dependency/FSURIHandler.java
<https://reviews.apache.org/r/25269/#comment100898>

FileSystem fs = getFileSystem(uri, conf, user);



core/src/main/java/org/apache/oozie/dependency/HCatURIHandler.java
<https://reviews.apache.org/r/25269/#comment100903>

HCatClient client = getHCatClient(uri, conf, user);
client.dropPartitions(hcatUri.getDb(), hcatUri.getTable(), 
hcatUri.getPartitionMap(), true);



core/src/main/java/org/apache/oozie/dependency/HCatURIHandler.java
<https://reviews.apache.org/r/25269/#comment100904>

Please change this method to getHCatClient(uri, conf, user) which only 
returns HCatClient;



core/src/main/java/org/apache/oozie/dependency/HCatURIHandler.java
<https://reviews.apache.org/r/25269/#comment100905>

Inline this method into getHCatClient()



core/src/main/java/org/apache/oozie/dependency/HCatURIHandler.java
<https://reviews.apache.org/r/25269/#comment100902>

Make this method as  getHCatClient(URI uri, Configuration conf) which 
retuns HCatClient for Oozie server and add new code in getHCatClient(URI uri, 
Configuration conf, String user)



core/src/main/java/org/apache/oozie/dependency/HCatURIHandler.java
<https://reviews.apache.org/r/25269/#comment100901>

Need to these changes. XLog.getLog(HCatURIHandler.class) is required to get 
the latest log prefix information.



core/src/main/java/org/apache/oozie/dependency/URIHandler.java
<https://reviews.apache.org/r/25269/#comment100900>

    * Delete a URI

 * @param uri URI


- Rohini Palaniswamy


On Oct. 31, 2014, 5:43 p.m., Ryota Egashira wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/25269/
> ---
> 
> (Updated Oct. 31, 2014, 5:43 p.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-1985
> https://issues.apache.org/jira/browse/OOZIE-1985
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> https://issues.apache.org/jira/browse/OOZIE-1985
> 
> 
> Diffs
> -
> 
>   core/src/main/java/org/apache/oozie/ErrorCode.java 4afeb6c 
>   core/src/main/java/org/apache/oozie/command/coord/CoordRerunXCommand.java 
> 098eb4b 
>   core/src/main/java/org/apache/oozie/dependency/FSURIHandler.java a8f548a 
>   core/src/main/java/org/apache/oozie/dependency/HCatURIHandler.java 629033b 
>   core/src/main/java/org/apache/oozie/dependency/URIHandler.java 7280902 
>   
> core/src/test/java/org/apache/oozie/command/coord/TestCoordRerunXCommand.java 
> 65338a3 
>   core/src/test/java/org/apache/oozie/dependency/TestHCatURIHandler.java 
> 31a68bf 
> 
> Diff: https://reviews.apache.org/r/25269/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Ryota Egashira
> 
>

Re: Review Request 27118: OOZIE-2045 Symlink support for sharelib

[Rohini Palaniswamy]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27118/#review59617
---



core/src/main/java/org/apache/oozie/service/ShareLibService.java
<https://reviews.apache.org/r/27118/#comment100914>

Does not throw URISyntaxException



core/src/main/java/org/apache/oozie/service/ShareLibService.java
<https://reviews.apache.org/r/27118/#comment100917>

Doesn't fs.isFile(rootDir) work? Why is the wrapping into URI required?



core/src/main/java/org/apache/oozie/service/ShareLibService.java
<https://reviews.apache.org/r/27118/#comment100919>

%s/symlink/Symlink/g

has changed



core/src/main/java/org/apache/oozie/service/ShareLibService.java
<https://reviews.apache.org/r/27118/#comment100920>

You need to update the particular key for sharelib name on the old map with 
value from new maps. You can't assign the new maps as they contain only value 
for the particular sharelib.



core/src/main/java/org/apache/oozie/service/ShareLibService.java
<https://reviews.apache.org/r/27118/#comment100935>

tmpShareLibMap



core/src/main/java/org/apache/oozie/service/ShareLibService.java
<https://reviews.apache.org/r/27118/#comment100926>

1) Either sharelibKey or sharelibName for the map key in all the places in 
code. Now there is a mix of actionKey, sharelibName and sharelibKey. 

2) Method javadoc also needs to include sharelibKey and need to mention 
only that particular sharelib will be loaded.



core/src/main/java/org/apache/oozie/service/ShareLibService.java
<https://reviews.apache.org/r/27118/#comment100936>

Can we pass ..,key.substring(SHARE_LIB_CONF_PREFIX.length() + 1), 
prop.getProperty(key).split(",")) instead of prop to the method changing the 
method signature to ...String sharelibKey, String[] sharelibPaths)?

Currently the new method assumes knowledge of internals of meta file. Would 
be cleaner to encapsulate that within this method.



core/src/main/java/org/apache/oozie/service/ShareLibService.java
<https://reviews.apache.org/r/27118/#comment100933>

Can remove else block and just add "Loading sharelib for " + key message to 
if block. Else there will be lot of log spam when symlinks are updated



core/src/main/java/org/apache/oozie/service/ShareLibService.java
<https://reviews.apache.org/r/27118/#comment100934>

Symlink



core/src/main/java/org/apache/oozie/service/ShareLibService.java
<https://reviews.apache.org/r/27118/#comment100929>

null and empty check for values() not required.



core/src/main/java/org/apache/oozie/service/ShareLibService.java
<https://reviews.apache.org/r/27118/#comment100931>

1) Use entryset instead of keyset.
2) Null and empty checks not required.



core/src/main/java/org/apache/oozie/service/ShareLibService.java
<https://reviews.apache.org/r/27118/#comment100932>

Have the code in try catch. Log any exception and return "Error retrieving 
symlink information".



core/src/test/java/org/apache/oozie/service/TestShareLibService.java
<https://reviews.apache.org/r/27118/#comment100922>

formatting



core/src/test/java/org/apache/oozie/service/TestShareLibService.java
<https://reviews.apache.org/r/27118/#comment100925>

createTestShareLibMetaFile


- Rohini Palaniswamy


On Oct. 31, 2014, 12:19 a.m., Purshotam Shah wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/27118/
> ---
> 
> (Updated Oct. 31, 2014, 12:19 a.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-2045
> https://issues.apache.org/jira/browse/OOZIE-2045
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> OOZIE-2045 Symlink support for sharelib
> 
> 
> Diffs
> -
> 
>   core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 
> 201cfa3 
>   core/src/main/java/org/apache/oozie/service/ShareLibService.java 63c5a41 
>   core/src/main/resources/oozie-default.xml 26eb7e0 
>   core/src/test/java/org/apache/oozie/service/TestShareLibService.java 
> 68cb357 
>   
> hadooplibs/hadoop-utils-0.23/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java
>  e69de29 
>   
> hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java
>  e69de29 
>   
> hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java
>  e69de29 
>   
> hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java
>  e69de29 
> 
> Diff: https://reviews.apache.org/r/27118/diff/
> 
> 
> Testing
> ---
> 
> UTC and manul testing
> 
> 
> Thanks,
> 
> Purshotam Shah
> 
>

Re: Review Request 27118: OOZIE-2045 Symlink support for sharelib

[Rohini Palaniswamy]

> On Nov. 3, 2014, 8:20 p.m., Rohini Palaniswamy wrote:
> > core/src/main/java/org/apache/oozie/service/ShareLibService.java, line 604
> > <https://reviews.apache.org/r/27118/diff/2/?file=745099#file745099line604>
> >
> > Can remove else block and just add "Loading sharelib for " + key 
> > message to if block. Else there will be lot of log spam when symlinks are 
> > updated
> 
> Purshotam Shah wrote:
> Incase of symlink, else will never execute because key will always prefix 
> with oozie.

This is still an issue. Please remove in the final patch


- Rohini


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27118/#review59617
---


On Nov. 6, 2014, 12:40 a.m., Purshotam Shah wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/27118/
> ---
> 
> (Updated Nov. 6, 2014, 12:40 a.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-2045
> https://issues.apache.org/jira/browse/OOZIE-2045
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> OOZIE-2045 Symlink support for sharelib
> 
> 
> Diffs
> -
> 
>   core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 
> 201cfa3 
>   core/src/main/java/org/apache/oozie/service/ShareLibService.java 63c5a41 
>   core/src/main/resources/oozie-default.xml 26eb7e0 
>   core/src/test/java/org/apache/oozie/service/TestShareLibService.java 
> 68cb357 
>   
> hadooplibs/hadoop-utils-0.23/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java
>  e69de29 
>   
> hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java
>  e69de29 
>   
> hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java
>  e69de29 
>   
> hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java
>  e69de29 
> 
> Diff: https://reviews.apache.org/r/27118/diff/
> 
> 
> Testing
> ---
> 
> UTC and manul testing
> 
> 
> Thanks,
> 
> Purshotam Shah
> 
>

Re: Review Request 27118: OOZIE-2045 Symlink support for sharelib

[Rohini Palaniswamy]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27118/#review60077
---

Ship it!


Just remove the Log message in the else part for the patch to be submitted in 
jira.

- Rohini Palaniswamy


On Nov. 6, 2014, 12:40 a.m., Purshotam Shah wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/27118/
> ---
> 
> (Updated Nov. 6, 2014, 12:40 a.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-2045
> https://issues.apache.org/jira/browse/OOZIE-2045
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> OOZIE-2045 Symlink support for sharelib
> 
> 
> Diffs
> -
> 
>   core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 
> 201cfa3 
>   core/src/main/java/org/apache/oozie/service/ShareLibService.java 63c5a41 
>   core/src/main/resources/oozie-default.xml 26eb7e0 
>   core/src/test/java/org/apache/oozie/service/TestShareLibService.java 
> 68cb357 
>   
> hadooplibs/hadoop-utils-0.23/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java
>  e69de29 
>   
> hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java
>  e69de29 
>   
> hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java
>  e69de29 
>   
> hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java
>  e69de29 
> 
> Diff: https://reviews.apache.org/r/27118/diff/
> 
> 
> Testing
> ---
> 
> UTC and manul testing
> 
> 
> Thanks,
> 
> Purshotam Shah
> 
>

Re: VOTE Release Oozie 4.1.0 (candidate 0)

[Rohini Palaniswamy]

Need to re-roll a new RC. We got an escalation today with a new 4.1
deployment for materialization not happening if frequency uses end of month
or days. This is a regression introduced by the changes that went in for
cron syntax for frequency. Works fine in older versions.

Puru,
   Could you put a patch once you have the fix ready?

-Rohini

On Wed, Nov 12, 2014 at 4:25 PM, bowen zhang <
bowenzhang...@yahoo.com.invalid> wrote:

> Hi folks,
> I have created a build for Oozie 4.1.0, candidate 0. Obviously, we have a
> lot of new features in 4.1 including but not limited to new sharelib
> service, consolidation of JPAExecutor, enhancement of HA with SLA and HCat,
> Improvement to Oozie coordinator ExecutionOrder. Keys to verify the
> signature of the release artifact are available at
> http://www.apache.org/dist/oozie/KEYS Please download, test, and try it
> out: http://people.apache.org/~/oozie-4.1.0-rc0 The
> release, md5 signature, gpg signature, and rat report can all
> be found at the above address.
> Vote closes on Saturday 5pm. But since we have a lot to test and Saturday
> is weekend, I am willing to extend the voting time beyond the conventional
> 72 hours to end of Next Monday.
>
> Bowen
>

Re: Review Request 25269: OOZIE-1985 support dropping hcat dataset in coord rerun with cleanup option

[Rohini Palaniswamy]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25269/#review61538
---



core/src/main/java/org/apache/oozie/dependency/HCatURIHandler.java
<https://reviews.apache.org/r/25269/#comment103233>

Add comment saying, read operations are allowed for all in hcat and so 
accessing as Oozie server itself. For write operations, doing doAs as user.



core/src/test/java/org/apache/oozie/command/coord/TestCoordRerunXCommand.java
<https://reviews.apache.org/r/25269/#comment103231>

keep the old method and overload with a new one.



core/src/test/java/org/apache/oozie/command/coord/TestCoordRerunXCommand.java
<https://reviews.apache.org/r/25269/#comment103230>

refresh



core/src/test/java/org/apache/oozie/command/coord/TestCoordRerunXCommand.java
<https://reviews.apache.org/r/25269/#comment103237>

Why do you need a transaction for read operation? Can you change to 
jpaexecutor service


- Rohini Palaniswamy


On Nov. 11, 2014, 6:26 p.m., Ryota Egashira wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/25269/
> ---
> 
> (Updated Nov. 11, 2014, 6:26 p.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-1985
> https://issues.apache.org/jira/browse/OOZIE-1985
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> https://issues.apache.org/jira/browse/OOZIE-1985
> 
> 
> Diffs
> -
> 
>   core/src/main/java/org/apache/oozie/ErrorCode.java 4afeb6c 
>   core/src/main/java/org/apache/oozie/command/coord/CoordRerunXCommand.java 
> 098eb4b 
>   core/src/main/java/org/apache/oozie/coord/CoordELFunctions.java 3bb191e 
>   core/src/main/java/org/apache/oozie/dependency/FSURIHandler.java a8f548a 
>   core/src/main/java/org/apache/oozie/dependency/HCatURIHandler.java 629033b 
>   core/src/main/java/org/apache/oozie/dependency/URIHandler.java 7280902 
>   
> core/src/test/java/org/apache/oozie/command/coord/TestCoordRerunXCommand.java 
> 65338a3 
>   core/src/test/java/org/apache/oozie/dependency/TestHCatURIHandler.java 
> 31a68bf 
> 
> Diff: https://reviews.apache.org/r/25269/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Ryota Egashira
> 
>

Re: VOTE Release Oozie 4.1.0 (candidate 1)

[Rohini Palaniswamy]

+1

On Tue, Nov 18, 2014 at 2:36 AM, Ryota Egashira <
egash...@yahoo-inc.com.invalid> wrote:

> +1
> -Built release artifact successfully
> -ran couple of example jobs on single-node setting, which succeeded.
>
> Ryota
>
> On 11/17/14, 5:10 PM, "Robert Kanter"  wrote:
>
> >+1
> >
> >- downloaded release artifacts
> >- built with Hadoop 1.1.1 (though ran against Hadoop 1.2.0) and ran a few
> >of the examples.
> >- built with Hadoop 2.3.0 (though ran against Hadoop some 2.7.0-SNAPSHOT)
> >and ran a few of the examples.  There's something wrong with my Hadoop 2
> >setup, but as far as I could tell, Oozie was working properly.
> >
> >- One thing: the release-log says that 4.1 is unrelated.  That should be
> >removed.  While you're at it, please also fix that for 4.0.1, I apparently
> >forgot to do that too :)
> >
> >- (And the key issue that was already mentioned should be fixed)
> >
> >
> >On Mon, Nov 17, 2014 at 11:18 AM, Mohammad Islam
> > >> wrote:
> >
> >> Sorry Bowen for the delay.I'm looking into this , I never tried that.
> >> Regards,Mohammad
> >>
> >>  On Monday, November 17, 2014 9:58 AM, bowen zhang
> >>  wrote:
> >>
> >>
> >>  Hi Mohammad,I think the KEY issue is the reason Mona has problem
> >>signing
> >> the page since I cannot commit my key to the repo. Can you resolve this
> >> ASAP?Thanks,Bowen
> >>
> >>   From: Mona Chitnis 
> >>  To: "dev@oozie.apache.org" ; bowen zhang <
> >> bowenzhang...@yahoo.com>
> >>  Sent: Monday, November 17, 2014 9:15 AM
> >>  Subject: Re: VOTE Release Oozie 4.1.0 (candidate 1)
> >>
> >> Downloaded the tarball, built and installed Oozie (-DskipTests) and ran
> >>an
> >> example M-R Oozie job against my Hadoop 2.4 cluster. +1 for that.
> >> Was unable to verify the md5 and gpg signatures. I dont find the key
> >>used
> >> for signing the page in the list of public keys on the KEYS page. Please
> >> let me know if I'm missing the right procedure.
> >> Regards, Mona Chitnis
> >>
> >>
> >>
> >>
> >> On Monday, November 17, 2014 8:08 AM, Shwetha GS <
> >> shwetha...@inmobi.com> wrote:
> >>
> >>
> >>  +1
> >>
> >> On Fri, Nov 14, 2014 at 6:49 AM, bowen zhang <
> >> bowenzhang...@yahoo.com.invalid> wrote:
> >>
> >> > Hi,
> >> >
> >> > I have created a build for Oozie 4.1.0, candidate 1.
> >> >
> >> > Keys to verify the signature of the release artifact are available at
> >> >
> >> >  http://www.apache.org/dist/oozie/KEYS
> >> >
> >> > Please download, test, and try it out:
> >> >
> >> >  http://people.apache.org/~bzhang/oozie-4.1.0-rc1
> >> >
> >> > The release, md5 signature, gpg signature, and rat report can all
> >> > be found at the above address.
> >> >
> >> > Vote closes on Monday EOD, the 17th.
> >> >
> >> > Bowen
> >> >
> >>
> >> --
> >> _
> >> The information contained in this communication is intended solely for
> >>the
> >> use of the individual or entity to whom it is addressed and others
> >> authorized to receive it. It may contain confidential or legally
> >>privileged
> >> information. If you are not the intended recipient you are hereby
> >>notified
> >> that any disclosure, copying, distribution or taking any action in
> >>reliance
> >> on the contents of this information is strictly prohibited and may be
> >> unlawful. If you have received this communication in error, please
> >>notify
> >> us immediately by responding to this email and then delete it from your
> >> system. The firm is neither liable for the proper and complete
> >>transmission
> >> of the information contained in this communication nor for any delay in
> >>its
> >> receipt.
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
>
>

Re: Review Request 27303: OOZIE-2035 NotificationXCommand should support proxy

[Rohini Palaniswamy]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27303/#review62308
---

Ship it!


Ship It!

- Rohini Palaniswamy


On Nov. 19, 2014, 11:08 p.m., Purshotam Shah wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/27303/
> ---
> 
> (Updated Nov. 19, 2014, 11:08 p.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-2035
> https://issues.apache.org/jira/browse/OOZIE-2035
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> NotificationXCommand should support proxy
> 
> 
> Diffs
> -
> 
>   client/src/main/java/org/apache/oozie/client/OozieClient.java 5e53a18 
>   core/src/main/java/org/apache/oozie/command/NotificationXCommand.java 
> e69de29 
>   
> core/src/main/java/org/apache/oozie/command/coord/CoordActionNotificationXCommand.java
>  2556152 
>   core/src/main/java/org/apache/oozie/command/wf/ActionKillXCommand.java 
> 6073e4d 
>   core/src/main/java/org/apache/oozie/command/wf/ActionStartXCommand.java 
> 373c942 
>   core/src/main/java/org/apache/oozie/command/wf/ActionXCommand.java 98fddfd 
>   core/src/main/java/org/apache/oozie/command/wf/KillXCommand.java cac1381 
>   core/src/main/java/org/apache/oozie/command/wf/ResumeXCommand.java 73c1447 
>   core/src/main/java/org/apache/oozie/command/wf/SignalXCommand.java 5ff0e5c 
>   core/src/main/java/org/apache/oozie/command/wf/SuspendXCommand.java 199af36 
>   
> core/src/main/java/org/apache/oozie/command/wf/WorkflowNotificationXCommand.java
>  0fc3d65 
>   core/src/main/resources/oozie-default.xml 19cae9d 
>   
> core/src/test/java/org/apache/oozie/command/coord/TestCoordActionNotificationXCommand.java
>  b58ecd9 
>   
> core/src/test/java/org/apache/oozie/command/wf/TestWorkflowNotificationXCommand.java
>  ad2fbd7 
>   docs/src/site/twiki/CoordinatorFunctionalSpec.twiki adf4ae1 
>   docs/src/site/twiki/WorkflowFunctionalSpec.twiki e6e52a0 
> 
> Diff: https://reviews.apache.org/r/27303/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Purshotam Shah
> 
>

Re: Review Request 25269: OOZIE-1985 support dropping hcat dataset in coord rerun with cleanup option

[Rohini Palaniswamy]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25269/#review62309
---

Ship it!


Ship It!

- Rohini Palaniswamy


On Nov. 19, 2014, 11:28 a.m., Ryota Egashira wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/25269/
> ---
> 
> (Updated Nov. 19, 2014, 11:28 a.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-1985
> https://issues.apache.org/jira/browse/OOZIE-1985
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> https://issues.apache.org/jira/browse/OOZIE-1985
> 
> 
> Diffs
> -
> 
>   core/src/main/java/org/apache/oozie/ErrorCode.java 4afeb6c 
>   core/src/main/java/org/apache/oozie/command/coord/CoordRerunXCommand.java 
> 098eb4b 
>   core/src/main/java/org/apache/oozie/coord/CoordELFunctions.java 3bb191e 
>   core/src/main/java/org/apache/oozie/dependency/FSURIHandler.java a8f548a 
>   core/src/main/java/org/apache/oozie/dependency/HCatURIHandler.java 629033b 
>   core/src/main/java/org/apache/oozie/dependency/URIHandler.java 7280902 
>   
> core/src/test/java/org/apache/oozie/command/coord/TestCoordRerunXCommand.java 
> ac023ca 
>   core/src/test/java/org/apache/oozie/dependency/TestHCatURIHandler.java 
> 31a68bf 
> 
> Diff: https://reviews.apache.org/r/25269/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Ryota Egashira
> 
>

Re: Review Request 28290: OOZIE-2070 reduce number of HCatalog access for coord latest check

[Rohini Palaniswamy]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28290/#review64100
---



core/src/main/java/org/apache/oozie/coord/CoordELFunctions.java
<https://reviews.apache.org/r/28290/#comment106524>

This is inefficient. Start with min(4, actual range count) as the size, 
then exponentially increase, but keep limit fixed at 200.



core/src/main/java/org/apache/oozie/dependency/FSURIHandler.java
<https://reviews.apache.org/r/28290/#comment106525>

Do not do this for fs. Unnecessary checks when not required if we are not 
using fs liststatus or globbing. Have a supportsBatchExists() method which will 
return false for FSUriHandler and if that is the case go with the old logic.



core/src/main/java/org/apache/oozie/dependency/HCatURIHandler.java
<https://reviews.apache.org/r/28290/#comment106533>

Need to use > or < when possible (MMDD, MMDDHH, etc patterns). OR 
is not efficient and will still put lot of load on hcat server



core/src/test/java/org/apache/oozie/dependency/TestHCatURIHandler.java
<https://reviews.apache.org/r/28290/#comment106551>

Have a extra partition "state" which is not used in the uri template. Users 
have a lot of partition, but mostly one use 1 or 2 in the uri template.


- Rohini Palaniswamy


On Nov. 20, 2014, 6:54 p.m., Ryota Egashira wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28290/
> ---
> 
> (Updated Nov. 20, 2014, 6:54 p.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-2070
> https://issues.apache.org/jira/browse/OOZIE-2070
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> https://issues.apache.org/jira/browse/OOZIE-2070
> 
> 
> Diffs
> -
> 
>   core/src/main/java/org/apache/oozie/coord/CoordELFunctions.java 7f59186 
>   core/src/main/java/org/apache/oozie/dependency/FSURIHandler.java 7c1aadf 
>   core/src/main/java/org/apache/oozie/dependency/HCatURIHandler.java 0e690a0 
>   core/src/main/java/org/apache/oozie/dependency/URIHandler.java 6e54d4b 
>   core/src/main/resources/oozie-default.xml 19cae9d 
>   
> core/src/test/java/org/apache/oozie/command/coord/TestCoordActionInputCheckXCommand.java
>  f79c9a0 
>   core/src/test/java/org/apache/oozie/dependency/TestFSURIHandler.java 
> 75d5429 
>   core/src/test/java/org/apache/oozie/dependency/TestHCatURIHandler.java 
> a49eba5 
>   core/src/test/java/org/apache/oozie/test/MiniHCatServer.java 8699ff8 
>   core/src/test/java/org/apache/oozie/test/XHCatTestCase.java 85ee1f2 
> 
> Diff: https://reviews.apache.org/r/28290/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Ryota Egashira
> 
>

Re: Review Request 28136: OOZIE-2068 - sharelib support for conf

[Rohini Palaniswamy]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28136/#review64448
---


In ShareLibService, call getShareLibFilesForActionConf() on each of the action 
executors and create a  HashSet of file names that will be added to action conf 
(currently only hive-site.xml). Whenever that file name is encountered, parse 
it and keep it in a local map 

In JavaActionExectuor, addShareLib

for (Path shareLibPath : shareLibPath) {
if 
(actionExecutor.getShareLibFilesForActionConf().contains(shareLibPath.getName())
 { //actually need to do fragment instead of name if fragment is present in uri
   addActionConf(ShareLibService.getConfiguration(shareLibPath));  //Do not 
overwrite in the action conf. Add only if not set as action conf has to take 
precedence.
} else {
  // add to distributed cache
}
}


core/src/main/java/org/apache/oozie/action/hadoop/Hive2ActionExecutor.java
<https://reviews.apache.org/r/28136/#comment107162>

List getShareLibFilesForActionConf



core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java
<https://reviews.apache.org/r/28136/#comment107163>

Not required. If fragment name or file name in 
getShareLibFileForActionConf, then read the conf file and add it to actionConf 
else add to distributed cache in addShareLib() itself.



core/src/main/java/org/apache/oozie/service/ShareLibService.java
<https://reviews.apache.org/r/28136/#comment107164>

No need to have separate tags for lib and conf.



core/src/test/java/org/apache/oozie/service/TestShareLibService.java
<https://reviews.apache.org/r/28136/#comment107176>

testConfFileAddedToActionConf



core/src/test/java/org/apache/oozie/service/TestShareLibService.java
<https://reviews.apache.org/r/28136/#comment107177>

testConfFileAddedToDistributedCache


- Rohini Palaniswamy


On Nov. 18, 2014, 9:10 a.m., Purshotam Shah wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28136/
> ---
> 
> (Updated Nov. 18, 2014, 9:10 a.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-2068
> https://issues.apache.org/jira/browse/OOZIE-2068
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> OOZIE-2068 - sharelib support for conf
> 
> 
> Diffs
> -
> 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2ActionExecutor.java 
> d70c3e1 
>   core/src/main/java/org/apache/oozie/action/hadoop/HiveActionExecutor.java 
> 832bbe6 
>   core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 
> fd1a98b 
>   core/src/main/java/org/apache/oozie/service/ShareLibService.java b408549 
>   core/src/main/resources/oozie-default.xml 19cae9d 
>   
> core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java 
> 48166a5 
>   core/src/test/java/org/apache/oozie/service/TestHAShareLibService.java 
> d2ad881 
>   core/src/test/java/org/apache/oozie/service/TestShareLibService.java 
> f261448 
>   tools/src/test/java/org/apache/oozie/tools/TestOozieSharelibCLI.java 
> a76014c 
> 
> Diff: https://reviews.apache.org/r/28136/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Purshotam Shah
> 
>

Re: Review Request 28637: OOZIE-2030 Configuration properties from global section is not getting set in Hadoop job conf when using sub-workflow action in Oozie workflow.xml

[Rohini Palaniswamy]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28637/#review64788
---



core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java
<https://reviews.apache.org/r/28637/#comment107555>

I think it actually makes sense and whatever I thought might conflict would 
be ok. Could you still add a config oozie.launcher.only.job.copy.actionconf (or 
some better name) as bother server and action level config with default value 
as true, so that we can turn it off in production if some unexpected conflict 
rises. If only few users encounter issue we can ask them to override at action 
level and if there is a major issue can turn it off at server level itself.



core/src/main/java/org/apache/oozie/action/oozie/SubWorkflowActionExecutor.java
<https://reviews.apache.org/r/28637/#comment107556>

Based on the code in LiteWorkflowAppParser.handleGlobal, 
eConf.getChild("configuration", ns) should contain the global configuration. 
Instead of parsing global and jobxml again, you could just use that.

  I think the cleaner way to do this would be to not touch ActionExecutors 
at all. Should parse and store global configuration in WorkflowInstance once 
and add it to each action configuration and also initialize global 
configuration of sub WorkflowInstance created with it. Currently the global 
section and its job.xml is parsed for each action which is really bad for 
latency. This would also take care of propagating it to nested sub-workflows. 
If the sub-workflow had global configuration then that should be merged while 
parsing itself to the parent global config initialized.


- Rohini Palaniswamy


On Dec. 8, 2014, 7:20 a.m., shwethags wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28637/
> ---
> 
> (Updated Dec. 8, 2014, 7:20 a.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-2030
> https://issues.apache.org/jira/browse/OOZIE-2030
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> 1. Propagate properties from configuration and job-xml from parent workflow 
> to sub-workflow. Currently, its propagated as workflow properties, but they 
> are required in action conf
> 2. For java actions, add properties from configuration and job-xml to 
> launcher job
> 
> 
> Diffs
> -
> 
>   core/src/main/java/org/apache/oozie/action/ActionExecutor.java ff836fb 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsActionExecutor.java 
> fed1d7a 
>   core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 
> f207d74 
>   
> core/src/main/java/org/apache/oozie/action/oozie/SubWorkflowActionExecutor.java
>  bda34b5 
>   core/src/main/java/org/apache/oozie/service/HadoopAccessorService.java 
> 18de48a 
>   
> core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java 
> 48166a5 
>   
> core/src/test/java/org/apache/oozie/action/oozie/TestSubWorkflowActionExecutor.java
>  0d7e926 
>   examples/src/main/apps/subwf/job.xml PRE-CREATION 
>   examples/src/main/apps/subwf/workflow.xml 9873a50 
> 
> Diff: https://reviews.apache.org/r/28637/diff/
> 
> 
> Testing
> ---
> 
> UTs and end to end test
> 
> 
> Thanks,
> 
> shwethags
> 
>

Re: Review Request 28637: OOZIE-2030 Configuration properties from global section is not getting set in Hadoop job conf when using sub-workflow action in Oozie workflow.xml

[Rohini Palaniswamy]

> On Dec. 11, 2014, 9:21 p.m., Rohini Palaniswamy wrote:
> > core/src/main/java/org/apache/oozie/action/oozie/SubWorkflowActionExecutor.java,
> >  line 180
> > <https://reviews.apache.org/r/28637/diff/3/?file=785338#file785338line180>
> >
> > Based on the code in LiteWorkflowAppParser.handleGlobal, 
> > eConf.getChild("configuration", ns) should contain the global 
> > configuration. Instead of parsing global and jobxml again, you could just 
> > use that.
> > 
> >   I think the cleaner way to do this would be to not touch 
> > ActionExecutors at all. Should parse and store global configuration in 
> > WorkflowInstance once and add it to each action configuration and also 
> > initialize global configuration of sub WorkflowInstance created with it. 
> > Currently the global section and its job.xml is parsed for each action 
> > which is really bad for latency. This would also take care of propagating 
> > it to nested sub-workflows. If the sub-workflow had global configuration 
> > then that should be merged while parsing itself to the parent global config 
> > initialized.
> 
> shwethags wrote:
> Makes sense. But that change will be more involved. Lets track that in 
> another jira

Just updating comment to the discussion I had with Shwetha. This will be the 
cleaner way of fixing this jira and cannot be done in another jira. Logically 
and with the current separation of work, it is not clean and crossing 
boundaries to parse workflow and job xml files for global section in 
ActionExecutors. They should be done during the actual parsing and construction 
of actions. ActionExecutors should only be for executing the action.


- Rohini


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28637/#review64788
---


On Dec. 8, 2014, 7:20 a.m., shwethags wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28637/
> ---
> 
> (Updated Dec. 8, 2014, 7:20 a.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-2030
> https://issues.apache.org/jira/browse/OOZIE-2030
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> 1. Propagate properties from configuration and job-xml from parent workflow 
> to sub-workflow. Currently, its propagated as workflow properties, but they 
> are required in action conf
> 2. For java actions, add properties from configuration and job-xml to 
> launcher job
> 
> 
> Diffs
> -
> 
>   core/src/main/java/org/apache/oozie/action/ActionExecutor.java ff836fb 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsActionExecutor.java 
> fed1d7a 
>   core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 
> f207d74 
>   
> core/src/main/java/org/apache/oozie/action/oozie/SubWorkflowActionExecutor.java
>  bda34b5 
>   core/src/main/java/org/apache/oozie/service/HadoopAccessorService.java 
> 18de48a 
>   
> core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java 
> 48166a5 
>   
> core/src/test/java/org/apache/oozie/action/oozie/TestSubWorkflowActionExecutor.java
>  0d7e926 
>   examples/src/main/apps/subwf/job.xml PRE-CREATION 
>   examples/src/main/apps/subwf/workflow.xml 9873a50 
> 
> Diff: https://reviews.apache.org/r/28637/diff/
> 
> 
> Testing
> ---
> 
> UTs and end to end test
> 
> 
> Thanks,
> 
> shwethags
> 
>

Action Required - Committers

[Rohini Palaniswamy]

Committers,
   I see a lot of resolved jiras without any Fix Version.

https://issues.apache.org/jira/issues/?jql=project%20%3D%20Oozie%20AND%20resolution%20%3D%20Fixed%20AND%20fixVersion%20is%20EMPTY%20ORDER%20BY%20updated%20DESC

  Can you please go and update the right Fix Version for the above jiras
for those that you committed. Please put the release branch they went into
if already released as part of a release before (4.0.0, 4.0.1 or 4.1.0) or
"trunk" if they are only in current trunk and is not released yet.


Regards,
Rohini

Re: Action Required - Committers

[Rohini Palaniswamy]

Thanks Robert. I think those should be fine without versions.

On Tue, Jan 6, 2015 at 10:40 AM, Robert Kanter  wrote:

> I updated the ones I was involved in.
>
> I did see that some of them are website, wiki, JIRA, maven, etc related,
> and those we should probably leave as "None" for the Fix Version.
> Alternatively, we could create a special Fix Version value for these sorts
> of things.
>
> On Tue, Jan 6, 2015 at 8:18 AM, Rohini Palaniswamy <
> rohini.adi...@gmail.com>
> wrote:
>
> > Committers,
> >I see a lot of resolved jiras without any Fix Version.
> >
> >
> >
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20Oozie%20AND%20resolution%20%3D%20Fixed%20AND%20fixVersion%20is%20EMPTY%20ORDER%20BY%20updated%20DESC
> >
> >   Can you please go and update the right Fix Version for the above jiras
> > for those that you committed. Please put the release branch they went
> into
> > if already released as part of a release before (4.0.0, 4.0.1 or 4.1.0)
> or
> > "trunk" if they are only in current trunk and is not released yet.
> >
> >
> > Regards,
> > Rohini
> >
>

Re: Review Request 28136: OOZIE-2068 - sharelib support for conf

[Rohini Palaniswamy]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28136/#review66957
---



core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java
<https://reviews.apache.org/r/28136/#comment110710>

String fragmentName = new URI(actionLibPath.toString()).getFragment();
String fileName = fragmentName == null ? path.getName() : fragmentName;



core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java
<https://reviews.apache.org/r/28136/#comment110695>

Check filename only if fragment name is null



core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java
<https://reviews.apache.org/r/28136/#comment110697>

action conf



core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java
<https://reviews.apache.org/r/28136/#comment110712>

Add comment
// Filtering out duplicate jars or files



core/src/main/java/org/apache/oozie/service/ShareLibService.java
<https://reviews.apache.org/r/28136/#comment110705>

Not used



core/src/main/java/org/apache/oozie/service/ShareLibService.java
<https://reviews.apache.org/r/28136/#comment110699>

LAUNCHER_LIB_PREFIX



core/src/main/java/org/apache/oozie/service/ShareLibService.java
<https://reviews.apache.org/r/28136/#comment111476>

Need not pass Set keyConfList and Map> shareLibConfigMap



core/src/main/java/org/apache/oozie/service/ShareLibService.java
<https://reviews.apache.org/r/28136/#comment111475>

fragment?



core/src/main/java/org/apache/oozie/service/ShareLibService.java
<https://reviews.apache.org/r/28136/#comment111477>

These can be done inside the getPathRecursively method and need not be 
passed as args



core/src/main/java/org/apache/oozie/service/ShareLibService.java
<https://reviews.apache.org/r/28136/#comment111479>

property.file -> config.file

Also this code should  be in the instrument method



core/src/main/java/org/apache/oozie/service/ShareLibService.java
<https://reviews.apache.org/r/28136/#comment111480>

Need to add the full paths



core/src/main/java/org/apache/oozie/service/ShareLibService.java
<https://reviews.apache.org/r/28136/#comment110708>

Why is this required?



core/src/main/java/org/apache/oozie/service/ShareLibService.java
<https://reviews.apache.org/r/28136/#comment111482>

Do this in initialize once and cache it as this would not change.



core/src/test/java/org/apache/oozie/service/TestShareLibService.java
<https://reviews.apache.org/r/28136/#comment111489>

We should make the last one take precedence



core/src/test/java/org/apache/oozie/service/TestShareLibService.java
<https://reviews.apache.org/r/28136/#comment111488>

third assert?



hadooplibs/hadoop-utils-0.23/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java
<https://reviews.apache.org/r/28136/#comment110703>

hasYARN()



hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java
<https://reviews.apache.org/r/28136/#comment111485>

why is this required? Add a comment to the code explaining why this is 
needed.


- Rohini Palaniswamy


On Jan. 5, 2015, 11:31 p.m., Purshotam Shah wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28136/
> ---
> 
> (Updated Jan. 5, 2015, 11:31 p.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-2068
> https://issues.apache.org/jira/browse/OOZIE-2068
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> OOZIE-2068 - sharelib support for conf
> 
> 
> Diffs
> -
> 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2ActionExecutor.java 
> d70c3e1 
>   core/src/main/java/org/apache/oozie/action/hadoop/HiveActionExecutor.java 
> 832bbe6 
>   core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 
> f207d74 
>   core/src/main/java/org/apache/oozie/service/ShareLibService.java bb0c7ed 
>   core/src/main/resources/oozie-default.xml 3d07c6f 
>   
> core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java 
> 48166a5 
>   core/src/test/java/org/apache/oozie/service/TestHAShareLibService.java 
> d2ad881 
>   core/src/test/java/org/apache/oozie/service/TestShareLibService.java 
> f261448 
>   docs/src/site/twiki/WorkflowFunctionalSpec.twiki 21199e8 
>   
> hadooplibs/hadoop-utils-0.23/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java
>  9a19770 
>   
> hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/hadoop/u

Re: Action Required - Committers

[Rohini Palaniswamy]

Thanks a lot Shwetha.  I hope you took care of jiras that went after 4.1
just into trunk.

Added the following step to "Create the Release Branch" section of
https://cwiki.apache.org/confluence/display/OOZIE/How+To+Release so that it
is not missed during the next release.

5. Do a search for all resolved jiras with fixVersion as trunk (JQL is
"project = Oozie AND resolution = Fixed AND fixVersion = trunk"). Bulk edit
them and change the fixVersion from trunk to the new branch version.

Regards,
Rohini


On Sun, Jan 11, 2015 at 10:12 PM, Shwetha G S  wrote:

> Did a bulk edit and fixed for all 255 jiras of 4.1 release. Looks like its
> already done for 4.0 and 4.0.1 releases
>
> -Shwetha
>
> On Sat, Jan 10, 2015 at 12:30 AM, Ryota Egashira <
> egash...@yahoo-inc.com.invalid> wrote:
>
> > just back from vacation. I did my part.Ryota
>

Re: Action Required - Committers

[Rohini Palaniswamy]

That's perfect Shwetha.

 The rest remaining without Fix Version are from Puru. Puru, Can you please
fix them?

Regards,
Rohini

On Tue, Jan 13, 2015 at 10:31 PM, Shwetha G S  wrote:

> I took the list of jiras from release-log.txt for 4.1.0. I didn't update
> any other jiras
>
> -Shwetha
>
> On Wed, Jan 14, 2015 at 4:48 AM, Rohini Palaniswamy <
> rohini.adi...@gmail.com
> > wrote:
>
> > Thanks a lot Shwetha.  I hope you took care of jiras that went after 4.1
> > just into trunk.
> >
> > Added the following step to "Create the Release Branch" section of
> > https://cwiki.apache.org/confluence/display/OOZIE/How+To+Release so that
> > it
> > is not missed during the next release.
> >
> > 5. Do a search for all resolved jiras with fixVersion as trunk (JQL is
> > "project = Oozie AND resolution = Fixed AND fixVersion = trunk"). Bulk
> edit
> > them and change the fixVersion from trunk to the new branch version.
> >
> > Regards,
> > Rohini
> >
> >
> > On Sun, Jan 11, 2015 at 10:12 PM, Shwetha G S 
> wrote:
> >
> > > Did a bulk edit and fixed for all 255 jiras of 4.1 release. Looks like
> > its
> > > already done for 4.0 and 4.0.1 releases
> > >
> > > -Shwetha
> > >
> > > On Sat, Jan 10, 2015 at 12:30 AM, Ryota Egashira <
> > > egash...@yahoo-inc.com.invalid> wrote:
> > >
> > > > just back from vacation. I did my part.Ryota
> > >
> >
>

Re: Review Request 28136: OOZIE-2068 - sharelib support for conf

[Rohini Palaniswamy]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28136/#review68455
---



core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java
<https://reviews.apache.org/r/28136/#comment112698>

path.toUri()



core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java
<https://reviews.apache.org/r/28136/#comment112694>

pathWithoutFragment 

path.toUri()



core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java
<https://reviews.apache.org/r/28136/#comment112699>

actionLibPath.toUri()



core/src/main/java/org/apache/oozie/service/ShareLibService.java
<https://reviews.apache.org/r/28136/#comment112696>

path.toUri()



core/src/main/java/org/apache/oozie/service/ShareLibService.java
<https://reviews.apache.org/r/28136/#comment112700>

Code should not depend on naming convention. Check against shareLibKey is 
redundant


- Rohini Palaniswamy


On Jan. 15, 2015, 10:48 p.m., Purshotam Shah wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28136/
> ---
> 
> (Updated Jan. 15, 2015, 10:48 p.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-2068
> https://issues.apache.org/jira/browse/OOZIE-2068
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> OOZIE-2068 - sharelib support for conf
> 
> 
> Diffs
> -
> 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2ActionExecutor.java 
> d70c3e1 
>   core/src/main/java/org/apache/oozie/action/hadoop/HiveActionExecutor.java 
> 832bbe6 
>   core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 
> f207d74 
>   core/src/main/java/org/apache/oozie/service/ShareLibService.java bb0c7ed 
>   core/src/main/resources/oozie-default.xml 3d07c6f 
>   
> core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java 
> 48166a5 
>   core/src/test/java/org/apache/oozie/service/TestHAShareLibService.java 
> d2ad881 
>   core/src/test/java/org/apache/oozie/service/TestShareLibService.java 
> f261448 
>   docs/src/site/twiki/WorkflowFunctionalSpec.twiki 21199e8 
>   
> hadooplibs/hadoop-utils-0.23/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java
>  9a19770 
>   
> hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java
>  bea4781 
>   
> hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java
>  acebd60 
>   
> hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java
>  acebd60 
>   tools/src/test/java/org/apache/oozie/tools/TestOozieSharelibCLI.java 
> a76014c 
> 
> Diff: https://reviews.apache.org/r/28136/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Purshotam Shah
> 
>

Re: Review Request 28136: OOZIE-2068 - sharelib support for conf

[Rohini Palaniswamy]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28136/#review68484
---



core/src/main/java/org/apache/oozie/service/ShareLibService.java
<https://reviews.apache.org/r/28136/#comment112752>

Set confList
 
- You need not maintain the action name


- Rohini Palaniswamy


On Jan. 15, 2015, 10:48 p.m., Purshotam Shah wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28136/
> ---
> 
> (Updated Jan. 15, 2015, 10:48 p.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-2068
> https://issues.apache.org/jira/browse/OOZIE-2068
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> OOZIE-2068 - sharelib support for conf
> 
> 
> Diffs
> -
> 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2ActionExecutor.java 
> d70c3e1 
>   core/src/main/java/org/apache/oozie/action/hadoop/HiveActionExecutor.java 
> 832bbe6 
>   core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 
> f207d74 
>   core/src/main/java/org/apache/oozie/service/ShareLibService.java bb0c7ed 
>   core/src/main/resources/oozie-default.xml 3d07c6f 
>   
> core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java 
> 48166a5 
>   core/src/test/java/org/apache/oozie/service/TestHAShareLibService.java 
> d2ad881 
>   core/src/test/java/org/apache/oozie/service/TestShareLibService.java 
> f261448 
>   docs/src/site/twiki/WorkflowFunctionalSpec.twiki 21199e8 
>   
> hadooplibs/hadoop-utils-0.23/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java
>  9a19770 
>   
> hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java
>  bea4781 
>   
> hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java
>  acebd60 
>   
> hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java
>  acebd60 
>   tools/src/test/java/org/apache/oozie/tools/TestOozieSharelibCLI.java 
> a76014c 
> 
> Diff: https://reviews.apache.org/r/28136/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Purshotam Shah
> 
>

Re: Review Request 28842: OOZIE-1913 Devise a way to turn off SLA alerts for bundle/coordinator flexibly

[Rohini Palaniswamy]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28842/#review68473
---


Review comments for Page 1 and documentation


client/src/main/java/org/apache/oozie/cli/OozieCLI.java
<https://reviews.apache.org/r/28842/#comment112714>

To be removed



client/src/main/java/org/apache/oozie/cli/OozieCLI.java
<https://reviews.apache.org/r/28842/#comment112741>

disables



client/src/main/java/org/apache/oozie/cli/OozieCLI.java
<https://reviews.apache.org/r/28842/#comment112742>

enables



client/src/main/java/org/apache/oozie/cli/OozieCLI.java
<https://reviews.apache.org/r/28842/#comment112729>

No value specified for -coordinator option



client/src/main/java/org/apache/oozie/client/OozieClient.java
<https://reviews.apache.org/r/28842/#comment112730>

Do not remove



client/src/main/java/org/apache/oozie/client/OozieClient.java
<https://reviews.apache.org/r/28842/#comment112731>

SLA_DISABLE_ALERT = disable.alerts



client/src/main/java/org/apache/oozie/client/OozieClient.java
<https://reviews.apache.org/r/28842/#comment112737>

coordId the coordinator job id
actionIds Comma separated list of action ids or action id ranges.
dates  Comma separated list of the nominal times

Param and javadoc change for other overloaded methods too.



client/src/main/java/org/apache/oozie/client/OozieClient.java
<https://reviews.apache.org/r/28842/#comment112733>

coordinators



client/src/main/java/org/apache/oozie/client/OozieClient.java
<https://reviews.apache.org/r/28842/#comment112738>

Give an example here



client/src/main/java/org/apache/oozie/client/event/jms/JMSHeaderConstants.java
<https://reviews.apache.org/r/28842/#comment112739>

Can we use jobId instead of coordId so that it can be used for workflows as 
well?



core/src/main/java/org/apache/oozie/BundleEngine.java
<https://reviews.apache.org/r/28842/#comment112743>

Rename classes to

BundleSLAAlertsEnableXCommand
BundleSLAAlertsDisableXCommand
BundleSLAChangeXCommand



core/src/main/java/org/apache/oozie/CoordinatorEngine.java
<https://reviews.apache.org/r/28842/#comment112744>

Rename classes to

CoordSLAAlertsEnableXCommand
CoordSLAAlertsDisableXCommand
CoordSLAChangeXCommand



core/src/main/java/org/apache/oozie/ErrorCode.java
<https://reviews.apache.org/r/28842/#comment112745>

failed:



core/src/main/java/org/apache/oozie/command/SlaAlertsXCommand.java
<https://reviews.apache.org/r/28842/#comment112750>

SLAXCommand



core/src/main/java/org/apache/oozie/command/bundle/BundleDisableSlaAlertsXCommand.java
<https://reviews.apache.org/r/28842/#comment112751>

Do we need two different classes in Bundle and Coord for enable and 
disable? Can't we just pass a boolean to the constructor ?



core/src/main/java/org/apache/oozie/command/coord/CoordChangeSlaAlertsXCommand.java
<https://reviews.apache.org/r/28842/#comment112756>

Please parse newParams  here and call slaChange with method args for each 
param (duration, start, end, etc)

Are EL functions evaluated for should-start, should-end, etc?



docs/src/site/twiki/DG_CommandLineTool.twiki
<https://reviews.apache.org/r/28842/#comment112727>

Make this a section and link into DG_SLAMonitoring.twiki

DG_SLAMonitoring.twiki:
  - Mention that alerting for 2 day old is not done.
   - Disabling while submission documention
   - Document the REST API as well



docs/src/site/twiki/DG_CommandLineTool.twiki
<https://reviews.apache.org/r/28842/#comment112723>

Eg:



docs/src/site/twiki/DG_CommandLineTool.twiki
<https://reviews.apache.org/r/28842/#comment112725>

Document the case where you disable or enable for all



docs/src/site/twiki/DG_CommandLineTool.twiki
<https://reviews.apache.org/r/28842/#comment112718>

1) sla_enable. _ is missing. Same for all other commands.

2) Remove spaces after , in action and date



docs/src/site/twiki/DG_CommandLineTool.twiki
<https://reviews.apache.org/r/28842/#comment112726>

Give this example with all values for -value



docs/src/site/twiki/DG_CommandLineTool.twiki
<https://reviews.apache.org/r/28842/#comment112719>

ending > is missing



webapp/src/main/webapp/console/sla/oozie-sla.html
<https://reviews.apache.org/r/28842/#comment112722>

Status can be removed to keep the column size small. Can keep 
slaAlertStatus in response as is.


- Rohini Palaniswamy


On Jan. 15, 2015, 2:01 a.m., Purshotam Shah wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28842/
&g

Re: Review Request 28136: OOZIE-2068 - sharelib support for conf

[Rohini Palaniswamy]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28136/#review68762
---

Ship it!


Ship It!

- Rohini Palaniswamy


On Jan. 20, 2015, 7:22 p.m., Purshotam Shah wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28136/
> ---
> 
> (Updated Jan. 20, 2015, 7:22 p.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-2068
> https://issues.apache.org/jira/browse/OOZIE-2068
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> OOZIE-2068 - sharelib support for conf
> 
> 
> Diffs
> -
> 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2ActionExecutor.java 
> d70c3e1 
>   core/src/main/java/org/apache/oozie/action/hadoop/HiveActionExecutor.java 
> 832bbe6 
>   core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 
> f207d74 
>   core/src/main/java/org/apache/oozie/service/ShareLibService.java bb0c7ed 
>   core/src/main/resources/oozie-default.xml 3d07c6f 
>   
> core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java 
> 48166a5 
>   core/src/test/java/org/apache/oozie/service/TestHAShareLibService.java 
> d2ad881 
>   core/src/test/java/org/apache/oozie/service/TestShareLibService.java 
> f261448 
>   docs/src/site/twiki/WorkflowFunctionalSpec.twiki 21199e8 
>   
> hadooplibs/hadoop-utils-0.23/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java
>  9a19770 
>   
> hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java
>  bea4781 
>   
> hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java
>  acebd60 
>   
> hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java
>  acebd60 
>   tools/src/test/java/org/apache/oozie/tools/TestOozieSharelibCLI.java 
> a76014c 
> 
> Diff: https://reviews.apache.org/r/28136/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Purshotam Shah
> 
>

Re: Review Request 30080: OOZIE-2118 add createdtime option to workflow jobs query

[Rohini Palaniswamy]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30080/#review68772
---


Can we have the commandline added as well.


client/src/main/java/org/apache/oozie/client/OozieClient.java
<https://reviews.apache.org/r/30080/#comment113202>

startcreatedtime for the commandline filter.

Need to create startCreatedTime in RESTConstants for WebServiceAPI.



core/src/main/java/org/apache/oozie/executor/jpa/WorkflowsJobGetJPAExecutor.java
<https://reviews.apache.org/r/30080/#comment113205>

equalsIgnoreCase



core/src/main/java/org/apache/oozie/executor/jpa/WorkflowsJobGetJPAExecutor.java
<https://reviews.apache.org/r/30080/#comment113201>

it should have only one value. Else you need to throw an exception. Same 
for end time



core/src/test/java/org/apache/oozie/executor/jpa/TestWorkflowsJobGetJPAExecutor.java
<https://reviews.apache.org/r/30080/#comment113209>

Remove the multiple ones



core/src/test/java/org/apache/oozie/executor/jpa/TestWorkflowsJobGetJPAExecutor.java
<https://reviews.apache.org/r/30080/#comment113210>

Remove the multiple ones



docs/src/site/twiki/WebServicesAPI.twiki
<https://reviews.apache.org/r/30080/#comment113206>

Add startcreatedtime and endcreatedtime to commandline documentation.


- Rohini Palaniswamy


On Jan. 20, 2015, 5:59 p.m., Ryota Egashira wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30080/
> ---
> 
> (Updated Jan. 20, 2015, 5:59 p.m.)
> 
> 
> Review request for oozie.
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> https://issues.apache.org/jira/browse/OOZIE-2118
> 
> 
> Diffs
> -
> 
>   client/src/main/java/org/apache/oozie/client/OozieClient.java 800e871 
>   core/src/main/java/org/apache/oozie/DagEngine.java 70ddd44 
>   
> core/src/main/java/org/apache/oozie/executor/jpa/WorkflowsJobGetJPAExecutor.java
>  5c0c0f9 
>   
> core/src/test/java/org/apache/oozie/executor/jpa/TestWorkflowsJobGetJPAExecutor.java
>  6250255 
>   core/src/test/java/org/apache/oozie/servlet/TestV1JobsServlet.java e84c0f0 
>   docs/src/site/twiki/WebServicesAPI.twiki fc51934 
> 
> Diff: https://reviews.apache.org/r/30080/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Ryota Egashira
> 
>

Re: Review Request 28842: OOZIE-1913 Devise a way to turn off SLA alerts for bundle/coordinator flexibly

[Rohini Palaniswamy]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28842/#review68784
---


Review for Page 2


core/src/main/java/org/apache/oozie/command/coord/CoordSlaAlertsXCommand.java
<https://reviews.apache.org/r/28842/#comment113226>

Why is this required and why do we get list of actions?



core/src/main/java/org/apache/oozie/command/coord/CoordSlaAlertsXCommand.java
<https://reviews.apache.org/r/28842/#comment113225>

Can you move this method to CoordUtils?



core/src/main/java/org/apache/oozie/coord/CoordUtils.java
<https://reviews.apache.org/r/28842/#comment113224>

getCoordActionIds



core/src/main/java/org/apache/oozie/coord/CoordUtils.java
<https://reviews.apache.org/r/28842/#comment113222>

isSLAAlertDisabled



core/src/main/java/org/apache/oozie/coord/CoordUtils.java
<https://reviews.apache.org/r/28842/#comment113223>

SLA_DISABLE_ALERT_OLDER_THAN



core/src/main/java/org/apache/oozie/coord/CoordUtils.java
<https://reviews.apache.org/r/28842/#comment113227>

SLA_DISABLE_ALERT



core/src/main/java/org/apache/oozie/coord/CoordUtils.java
<https://reviews.apache.org/r/28842/#comment113228>

What about ids?



core/src/main/java/org/apache/oozie/coord/CoordUtils.java
<https://reviews.apache.org/r/28842/#comment113229>

Search for suspend and rename to disabled everywhere



core/src/main/java/org/apache/oozie/coord/CoordUtils.java
<https://reviews.apache.org/r/28842/#comment113230>

getCoordActionSLAAlertStatus



core/src/main/java/org/apache/oozie/coord/CoordUtils.java
<https://reviews.apache.org/r/28842/#comment113231>

why is this block required? Is this not being checked in the first block in 
the method?



core/src/main/java/org/apache/oozie/coord/CoordUtils.java
<https://reviews.apache.org/r/28842/#comment113232>

slaAlertConfig



core/src/main/java/org/apache/oozie/coord/CoordUtils.java
<https://reviews.apache.org/r/28842/#comment113233>

could be enable as well



core/src/main/java/org/apache/oozie/coord/CoordUtils.java
<https://reviews.apache.org/r/28842/#comment113235>

value : values



core/src/main/java/org/apache/oozie/coord/CoordUtils.java
<https://reviews.apache.org/r/28842/#comment113234>

what if it was not action range and just actions? For eg: 1,5,7



core/src/main/java/org/apache/oozie/jms/JMSSLAEventListener.java
<https://reviews.apache.org/r/28842/#comment113245>

What is this for and why is it needed?



core/src/main/java/org/apache/oozie/servlet/V0JobServlet.java
<https://reviews.apache.org/r/28842/#comment113247>

v0. In other methods as well



core/src/main/java/org/apache/oozie/servlet/V2SLAServlet.java
<https://reviews.apache.org/r/28842/#comment113252>

LASLARegistrationMap -> slaConfigMap



core/src/main/java/org/apache/oozie/sla/SLACalculator.java
<https://reviews.apache.org/r/28842/#comment113254>

Map newParams - Please use the exact method parameters 
instead of passing a map



core/src/main/java/org/apache/oozie/sla/SLACalculatorMemory.java
<https://reviews.apache.org/r/28842/#comment113257>

Can you extract this code to a private method? It is repeated multiple 
times throughout.



core/src/main/java/org/apache/oozie/sla/SLACalculatorMemory.java
<https://reviews.apache.org/r/28842/#comment113255>

This is an unnecessary update. Should be only done when there is a change.



core/src/main/java/org/apache/oozie/sla/SLACalculatorMemory.java
<https://reviews.apache.org/r/28842/#comment113261>

enableAlert(List jobIds, List parentIds)

Same for other methods



core/src/main/java/org/apache/oozie/sla/SLARegistrationBean.java
<https://reviews.apache.org/r/28842/#comment113237>

GET_SLA_CONFIG



core/src/main/java/org/apache/oozie/sla/SLASummaryBean.java
<https://reviews.apache.org/r/28842/#comment113264>

slaConfigMap



core/src/main/java/org/apache/oozie/sla/SLASummaryBean.java
<https://reviews.apache.org/r/28842/#comment113262>

Disabled , Enabled


- Rohini Palaniswamy


On Jan. 15, 2015, 2:01 a.m., Purshotam Shah wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28842/
> ---
> 
> (Updated Jan. 15, 2015, 2:01 a.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-1913
> https://issues.apache.org/jira/browse/OOZIE-1913
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> OOZIE-1913 Devise a way to turn off SLA alerts for bundle/coordinator flexibly
> 
> 
> Diffs
> -
> 
>   cl

Re: Review Request 28842: OOZIE-1913 Devise a way to turn off SLA alerts for bundle/coordinator flexibly

[Rohini Palaniswamy]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28842/#review68798
---


Review request for Page 3


client/src/main/java/org/apache/oozie/client/rest/RestConstants.java
<https://reviews.apache.org/r/28842/#comment113278>

Remove NEW from the constant names



core/src/main/java/org/apache/oozie/sla/service/SLAService.java
<https://reviews.apache.org/r/28842/#comment113268>

Not required. Same message inside updateAllSlaStatus()



core/src/main/java/org/apache/oozie/sla/service/SLAService.java
<https://reviews.apache.org/r/28842/#comment113270>

API Changes for the new methods as suggested same as SLACalculator



core/src/main/resources/oozie-default.xml
<https://reviews.apache.org/r/28842/#comment113271>

disable



core/src/test/java/org/apache/oozie/command/TestSlaAlertCommand.java
<https://reviews.apache.org/r/28842/#comment113276>

TestSLAXCommand



core/src/test/java/org/apache/oozie/command/TestSlaAlertCommand.java
<https://reviews.apache.org/r/28842/#comment113277>

testBundle



core/src/test/java/org/apache/oozie/command/coord/TestCoordSubmitXCommand.java
<https://reviews.apache.org/r/28842/#comment113280>

Good test coverage. But all this info (configs while submitting jobs) is 
missing in documentation. Need to add it there.



core/src/test/java/org/apache/oozie/coord/TestCoordUtils.java
<https://reviews.apache.org/r/28842/#comment113282>

There seems to be no code change in this class. Can be reverted



core/src/test/java/org/apache/oozie/jms/TestJMSSLAEventListener.java
<https://reviews.apache.org/r/28842/#comment113284>

JMSHeaderConstants.COORD_ID not required at this point and can be reverted. 
Selectors add load to ActiveMQ servers. If there is actually a feature ask we 
can add it then.


- Rohini Palaniswamy


On Jan. 15, 2015, 2:01 a.m., Purshotam Shah wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28842/
> ---
> 
> (Updated Jan. 15, 2015, 2:01 a.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-1913
> https://issues.apache.org/jira/browse/OOZIE-1913
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> OOZIE-1913 Devise a way to turn off SLA alerts for bundle/coordinator flexibly
> 
> 
> Diffs
> -
> 
>   client/src/main/java/org/apache/oozie/cli/OozieCLI.java bc8fa50 
>   client/src/main/java/org/apache/oozie/client/OozieClient.java 800e871 
>   
> client/src/main/java/org/apache/oozie/client/event/jms/JMSHeaderConstants.java
>  801ad7e 
>   client/src/main/java/org/apache/oozie/client/rest/JsonTags.java b7cf0e7 
>   client/src/main/java/org/apache/oozie/client/rest/RestConstants.java 
> 85efecf 
>   core/src/main/java/org/apache/oozie/BaseEngine.java a982ad4 
>   core/src/main/java/org/apache/oozie/BundleEngine.java 27afe73 
>   core/src/main/java/org/apache/oozie/CoordinatorEngine.java 3406c6f 
>   core/src/main/java/org/apache/oozie/CoordinatorJobBean.java 4d6b970 
>   core/src/main/java/org/apache/oozie/DagEngine.java 70ddd44 
>   core/src/main/java/org/apache/oozie/ErrorCode.java c87 
>   core/src/main/java/org/apache/oozie/command/SlaAlertsXCommand.java e69de29 
>   
> core/src/main/java/org/apache/oozie/command/bundle/BundleChangeSlaAlertsXCommand.java
>  e69de29 
>   
> core/src/main/java/org/apache/oozie/command/bundle/BundleDisableSlaAlertsXCommand.java
>  e69de29 
>   
> core/src/main/java/org/apache/oozie/command/bundle/BundleEnableSlaAlertsXCommand.java
>  e69de29 
>   
> core/src/main/java/org/apache/oozie/command/bundle/BundleSlaAlertsXCommand.java
>  e69de29 
>   
> core/src/main/java/org/apache/oozie/command/coord/CoordChangeSlaAlertsXCommand.java
>  e69de29 
>   
> core/src/main/java/org/apache/oozie/command/coord/CoordDisableSlaAlertsXCommand.java
>  e69de29 
>   
> core/src/main/java/org/apache/oozie/command/coord/CoordEnableSlaAlertsXCommand.java
>  e69de29 
>   
> core/src/main/java/org/apache/oozie/command/coord/CoordMaterializeTransitionXCommand.java
>  548946f 
>   
> core/src/main/java/org/apache/oozie/command/coord/CoordSlaAlertsXCommand.java 
> e69de29 
>   core/src/main/java/org/apache/oozie/coord/CoordUtils.java 4643d73 
>   core/src/main/java/org/apache/oozie/executor/jpa/CoordJobQueryExecutor.java 
> 4bccef4 
>   
> core/src/main/java/org/apache/oozie/executor/jpa/SLARegistrationQueryExecutor.java
>  e220c01 
>   
> core/src/main/java/org/apache/oozie/executor/jpa/SL

Re: Review Request 30080: OOZIE-2118 add createdtime option to workflow jobs query

[Rohini Palaniswamy]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30080/#review68927
---

Ship it!


Please make the below minor changes in the final patch uploaded to jira


core/src/main/java/org/apache/oozie/executor/jpa/WorkflowsJobGetJPAExecutor.java
<https://reviews.apache.org/r/30080/#comment113447>

startcreatedtime



core/src/main/java/org/apache/oozie/executor/jpa/WorkflowsJobGetJPAExecutor.java
<https://reviews.apache.org/r/30080/#comment113448>

endcreatedtime


- Rohini Palaniswamy


On Jan. 21, 2015, 12:27 a.m., Ryota Egashira wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30080/
> ---
> 
> (Updated Jan. 21, 2015, 12:27 a.m.)
> 
> 
> Review request for oozie.
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> https://issues.apache.org/jira/browse/OOZIE-2118
> 
> 
> Diffs
> -
> 
>   client/src/main/java/org/apache/oozie/cli/OozieCLI.java bc8fa50 
>   client/src/main/java/org/apache/oozie/client/OozieClient.java 800e871 
>   core/src/main/java/org/apache/oozie/DagEngine.java 70ddd44 
>   
> core/src/main/java/org/apache/oozie/executor/jpa/WorkflowsJobGetJPAExecutor.java
>  5c0c0f9 
>   core/src/test/java/org/apache/oozie/client/TestOozieCLI.java ac54b09 
>   
> core/src/test/java/org/apache/oozie/executor/jpa/TestWorkflowsJobGetJPAExecutor.java
>  6250255 
>   core/src/test/java/org/apache/oozie/servlet/TestV1JobsServlet.java e84c0f0 
>   docs/src/site/twiki/DG_CommandLineTool.twiki 4ac73f9 
>   docs/src/site/twiki/WebServicesAPI.twiki fc51934 
> 
> Diff: https://reviews.apache.org/r/30080/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Ryota Egashira
> 
>

Re: Review Request 30080: OOZIE-2118 add createdtime option to workflow jobs query

[Rohini Palaniswamy]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30080/#review69213
---



core/src/main/java/org/apache/oozie/executor/jpa/WorkflowsJobGetJPAExecutor.java
<https://reviews.apache.org/r/30080/#comment113827>

Could you also add m for minutes. d and h - only support lower as that is 
the conventional date format. If someone wants to add M later they can do for 
month.



core/src/test/java/org/apache/oozie/executor/jpa/TestWorkflowsJobGetJPAExecutor.java
<https://reviews.apache.org/r/30080/#comment113829>

Can we also assert the actual created time in the workflows returned.


- Rohini Palaniswamy


On Jan. 22, 2015, 3:02 a.m., Ryota Egashira wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30080/
> ---
> 
> (Updated Jan. 22, 2015, 3:02 a.m.)
> 
> 
> Review request for oozie.
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> https://issues.apache.org/jira/browse/OOZIE-2118
> 
> 
> Diffs
> -
> 
>   client/src/main/java/org/apache/oozie/cli/OozieCLI.java bc8fa50 
>   client/src/main/java/org/apache/oozie/client/OozieClient.java 800e871 
>   core/src/main/java/org/apache/oozie/DagEngine.java 70ddd44 
>   
> core/src/main/java/org/apache/oozie/executor/jpa/WorkflowsJobGetJPAExecutor.java
>  5c0c0f9 
>   core/src/test/java/org/apache/oozie/client/TestOozieCLI.java ac54b09 
>   
> core/src/test/java/org/apache/oozie/executor/jpa/TestWorkflowsJobGetJPAExecutor.java
>  6250255 
>   core/src/test/java/org/apache/oozie/servlet/TestV1JobsServlet.java e84c0f0 
>   docs/src/site/twiki/DG_CommandLineTool.twiki 4ac73f9 
>   docs/src/site/twiki/WebServicesAPI.twiki fc51934 
> 
> Diff: https://reviews.apache.org/r/30080/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Ryota Egashira
> 
>

Re: Review Request 30240: OOZIE-2123 Disable launcher uber mode if classloader options are set

[Rohini Palaniswamy]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30240/#review69664
---

Ship it!


Ship It!

- Rohini Palaniswamy


On Jan. 24, 2015, 1:49 a.m., Ryota Egashira wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30240/
> ---
> 
> (Updated Jan. 24, 2015, 1:49 a.m.)
> 
> 
> Review request for oozie.
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> https://issues.apache.org/jira/browse/OOZIE-2123
> 
> 
> Diffs
> -
> 
>   core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 
> f207d74 
>   
> core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java 
> 48166a5 
> 
> Diff: https://reviews.apache.org/r/30240/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Ryota Egashira
> 
>

Re: Review Request 30080: OOZIE-2118 add createdtime option to workflow jobs query

[Rohini Palaniswamy]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30080/#review69944
---

Ship it!


Ship It!

- Rohini Palaniswamy


On Jan. 27, 2015, 1:35 a.m., Ryota Egashira wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30080/
> ---
> 
> (Updated Jan. 27, 2015, 1:35 a.m.)
> 
> 
> Review request for oozie.
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> https://issues.apache.org/jira/browse/OOZIE-2118
> 
> 
> Diffs
> -
> 
>   client/src/main/java/org/apache/oozie/cli/OozieCLI.java bc8fa50 
>   client/src/main/java/org/apache/oozie/client/OozieClient.java 800e871 
>   core/src/main/java/org/apache/oozie/DagEngine.java 70ddd44 
>   
> core/src/main/java/org/apache/oozie/executor/jpa/WorkflowsJobGetJPAExecutor.java
>  5c0c0f9 
>   core/src/main/java/org/apache/oozie/util/XLogUserFilterParam.java af9b979 
>   core/src/test/java/org/apache/oozie/client/TestOozieCLI.java ac54b09 
>   
> core/src/test/java/org/apache/oozie/executor/jpa/TestWorkflowsJobGetJPAExecutor.java
>  6250255 
>   core/src/test/java/org/apache/oozie/servlet/TestV1JobsServlet.java e84c0f0 
>   docs/src/site/twiki/DG_CommandLineTool.twiki 4ac73f9 
>   docs/src/site/twiki/WebServicesAPI.twiki fc51934 
> 
> Diff: https://reviews.apache.org/r/30080/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Ryota Egashira
> 
>

Re: Review Request 28842: OOZIE-1913 Devise a way to turn off SLA alerts for bundle/coordinator flexibly

[Rohini Palaniswamy]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28842/#review69707
---


There is still a "suspend" instead of disable in DG_CommandLineTool.twiki


client/src/main/java/org/apache/oozie/client/OozieClient.java
<https://reviews.apache.org/r/28842/#comment115103>

Change SLA definition of a coordinator
SLA change definition parameters can be..
Supported parameter key names are ...

Change to be done on the overloaded method as well.



core/src/main/java/org/apache/oozie/sla/SLACalculatorMemory.java
<https://reviews.apache.org/r/28842/#comment115119>

Once changed, don't you have to reload the changed values in memory?

For eg: If expected endtime was changed from 30mins after nominal time to 
10 mins after nominal time, an alert would not be sent at 11th min as memory 
still has 30 mins and is checking against that.



core/src/main/java/org/apache/oozie/sla/SLAOperations.java
<https://reviews.apache.org/r/28842/#comment115122>

add instead of ddd



core/src/test/java/org/apache/oozie/sla/TestSLAEventGeneration.java
<https://reviews.apache.org/r/28842/#comment114430>

To be removed



docs/src/site/twiki/DG_CommandLineTool.twiki
<https://reviews.apache.org/r/28842/#comment115130>

Changing job SLA definition and alerting

You need to define a anchor tag #JobSLACommand here to actually reference 
it from other pages. For eg, refer #HCatPigExampleOne in 
CoordinatorFunctionSpec twiki.



docs/src/site/twiki/DG_CommandLineTool.twiki
<https://reviews.apache.org/r/28842/#comment115125>

actions



core/src/main/java/org/apache/oozie/CoordinatorActionBean.java
<https://reviews.apache.org/r/28842/#comment115108>

Can the queries be re-written to say a.statusStr in 
('PREP','RUNNING',..) so that the status index will be used.



core/src/main/java/org/apache/oozie/command/coord/CoordSLAAlertsEnableXCommand.java
<https://reviews.apache.org/r/28842/#comment115114>

SLA.alerts.enable/disable



docs/src/site/twiki/DG_SLAMonitoring.twiki
<https://reviews.apache.org/r/28842/#comment115126>

Changing job SLA definition and alerting



docs/src/site/twiki/DG_SLAMonitoring.twiki
<https://reviews.apache.org/r/28842/#comment115129>

SLA alert status can also be modified through commandline or REST API after 
submission for running jobs. Refer [[DG_CommandLineTool#JobSLACommand][job -sla 
command]] for commandline usage. Refer the REST API [[..][here]].

SLA definition of should-start, should-end, nominal-time and max-duration 
can be changed for running jobs through commandline or REST API. 
Refer [[DG_CommandLineTool#JobSLACommand][job -sla command]] for 
commandline usage. Refer the REST API [[..][here]].



docs/src/site/twiki/WebServicesAPI.twiki
<https://reviews.apache.org/r/28842/#comment115132>

Changing job SLA definition and alerting



docs/src/site/twiki/WebServicesAPI.twiki
<https://reviews.apache.org/r/28842/#comment115133>

Need to document all key options



docs/src/site/twiki/WebServicesAPI.twiki
<https://reviews.apache.org/r/28842/#comment115134>

Need to document date and coordinator options for bundle.


- Rohini Palaniswamy


On Jan. 26, 2015, 11:14 p.m., Purshotam Shah wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28842/
> ---
> 
> (Updated Jan. 26, 2015, 11:14 p.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-1913
> https://issues.apache.org/jira/browse/OOZIE-1913
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> OOZIE-1913 Devise a way to turn off SLA alerts for bundle/coordinator flexibly
> 
> 
> Diffs
> -
> 
>   client/src/main/java/org/apache/oozie/cli/OozieCLI.java bc8fa50 
>   client/src/main/java/org/apache/oozie/client/OozieClient.java 800e871 
>   
> client/src/main/java/org/apache/oozie/client/event/jms/JMSHeaderConstants.java
>  801ad7e 
>   client/src/main/java/org/apache/oozie/client/rest/JsonTags.java b7cf0e7 
>   client/src/main/java/org/apache/oozie/client/rest/RestConstants.java 
> 85efecf 
>   core/src/main/java/org/apache/oozie/BaseEngine.java a982ad4 
>   core/src/main/java/org/apache/oozie/BundleEngine.java 27afe73 
>   core/src/main/java/org/apache/oozie/CoordinatorActionBean.java bd01d14 
>   core/src/main/java/org/apache/oozie/CoordinatorEngine.java 3406c6f 
>   core/src/main/java/org/apache/oozie/CoordinatorJobBean.java 4d6b970 
>   core/src/main/java/org

Re: Review Request 28842: OOZIE-1913 Devise a way to turn off SLA alerts for bundle/coordinator flexibly

[Rohini Palaniswamy]

> On Jan. 16, 2015, 10:28 p.m., Rohini Palaniswamy wrote:
> > client/src/main/java/org/apache/oozie/client/OozieClient.java, lines 
> > 1613-1615
> > <https://reviews.apache.org/r/28842/diff/3/?file=822303#file822303line1613>
> >
> > coordId the coordinator job id
> > actionIds Comma separated list of action ids or action id ranges.
> > dates  Comma separated list of the nominal times
> > 
> > Param and javadoc change for other overloaded methods too.
> 
> Purshotam Shah wrote:
> >coordId the coordinator job id
> JobId can be bundle id also without coord name/id.
> @param jobIds the job ids is correct.

Since you have another API with first one as bundleId, make this as the 
coordId. Add a @Nullable to coords.


> On Jan. 16, 2015, 10:28 p.m., Rohini Palaniswamy wrote:
> > core/src/main/java/org/apache/oozie/command/SlaAlertsXCommand.java, line 25
> > <https://reviews.apache.org/r/28842/diff/3/?file=822313#file822313line25>
> >
> > SLAXCommand
> 
> Purshotam Shah wrote:
> SLAXCommand  will be confusing name, there are other SLACommand and 
> SlaAlertsXCommand is only for sla alertcommand. So SLAXCommand  will 
> incorrect and give wrong meaning.

No problem. But can we make capitalize SLA. i.e SLAAlertsXCommand


> On Jan. 16, 2015, 10:28 p.m., Rohini Palaniswamy wrote:
> > core/src/main/java/org/apache/oozie/command/bundle/BundleDisableSlaAlertsXCommand.java,
> >  line 24
> > <https://reviews.apache.org/r/28842/diff/3/?file=822315#file822315line24>
> >
> > Do we need two different classes in Bundle and Coord for enable and 
> > disable? Can't we just pass a boolean to the constructor ?
> 
> Purshotam Shah wrote:
> Yes, both classes have different implementation.  Bundle loads bundle job 
> find out all coords and call coordCommand to perform SLA operation.

Not asking to combine bundle and coord commands. But asking to combine 
BundleSLAAlertDisableXCommand and BundleSLAAlertEnableXCommand as there is 
duplication of code. Same for coordinator.


> On Jan. 16, 2015, 10:28 p.m., Rohini Palaniswamy wrote:
> > client/src/main/java/org/apache/oozie/client/event/jms/JMSHeaderConstants.java,
> >  line 34
> > <https://reviews.apache.org/r/28842/diff/3/?file=822304#file822304line34>
> >
> > Can we use jobId instead of coordId so that it can be used for 
> > workflows as well?

Needs to be removed


- Rohini


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28842/#review68473
---


On Jan. 26, 2015, 11:14 p.m., Purshotam Shah wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28842/
> ---
> 
> (Updated Jan. 26, 2015, 11:14 p.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-1913
> https://issues.apache.org/jira/browse/OOZIE-1913
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> OOZIE-1913 Devise a way to turn off SLA alerts for bundle/coordinator flexibly
> 
> 
> Diffs
> -
> 
>   client/src/main/java/org/apache/oozie/cli/OozieCLI.java bc8fa50 
>   client/src/main/java/org/apache/oozie/client/OozieClient.java 800e871 
>   
> client/src/main/java/org/apache/oozie/client/event/jms/JMSHeaderConstants.java
>  801ad7e 
>   client/src/main/java/org/apache/oozie/client/rest/JsonTags.java b7cf0e7 
>   client/src/main/java/org/apache/oozie/client/rest/RestConstants.java 
> 85efecf 
>   core/src/main/java/org/apache/oozie/BaseEngine.java a982ad4 
>   core/src/main/java/org/apache/oozie/BundleEngine.java 27afe73 
>   core/src/main/java/org/apache/oozie/CoordinatorActionBean.java bd01d14 
>   core/src/main/java/org/apache/oozie/CoordinatorEngine.java 3406c6f 
>   core/src/main/java/org/apache/oozie/CoordinatorJobBean.java 4d6b970 
>   core/src/main/java/org/apache/oozie/DagEngine.java 70ddd44 
>   core/src/main/java/org/apache/oozie/ErrorCode.java c87 
>   core/src/main/java/org/apache/oozie/command/SlaAlertsXCommand.java e69de29 
>   
> core/src/main/java/org/apache/oozie/command/bundle/BundleSLAAlertsDisableXCommand.java
>  e69de29 
>   
> core/src/main/java/org/apache/oozie/command/bundle/BundleSLAAlertsEnableXCommand.java
>  e69de29 
>   
> core/src/main/java/org/apache/oozie/command/bundle/BundleSLAChangeXCommand.java
>  e69de29 
>   
> core/src/main/java/org/apache/oozie/command/bundle/BundleSlaAle

Re: Review Request 28842: OOZIE-1913 Devise a way to turn off SLA alerts for bundle/coordinator flexibly

[Rohini Palaniswamy]

> On Jan. 20, 2015, 10:34 p.m., Rohini Palaniswamy wrote:
> > core/src/main/java/org/apache/oozie/coord/CoordUtils.java, line 324
> > <https://reviews.apache.org/r/28842/diff/3/?file=822323#file822323line324>
> >
> > could be enable as well
> 
> Purshotam Shah wrote:
> yes disable overrides enable.

Just rename the variable slaDisableAlert to slaAlertValue or something as it is 
misleading as the slaAlertType  could be enable as well


> On Jan. 20, 2015, 10:34 p.m., Rohini Palaniswamy wrote:
> > core/src/main/java/org/apache/oozie/sla/SLARegistrationBean.java, line 59
> > <https://reviews.apache.org/r/28842/diff/3/?file=822340#file822340line59>
> >
> > GET_SLA_CONFIG
> 
> Purshotam Shah wrote:
> I guess GET_SLA_CONFIG_BULK  is better name. It indicate that SQL will 
> accept multiple jobids. When we pass single job, GET_SLA_CONFIG looks 
> relevant, but in this case GET_SLA_CONFIG_BULK is a proper name.
> 
> @NamedQuery(name = "GET_SLA_CONFIG_BULK", query = "select w.jobId, 
> w.slaConfig from SLARegistrationBean w where w.jobId IN (:ids)"),
> @NamedQuery(name = "GET_SLA_CONFIG", query = "select w.jobId, w.slaConfig 
> from SLARegistrationBean w where w.jobId =:id")

Bulk is bad naming and is not used in analytics terminology for fetching more 
than one row. Change it to GET_SLA_CONFIGS.


- Rohini


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28842/#review68784
---


On Jan. 26, 2015, 11:14 p.m., Purshotam Shah wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28842/
> ---
> 
> (Updated Jan. 26, 2015, 11:14 p.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-1913
> https://issues.apache.org/jira/browse/OOZIE-1913
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> OOZIE-1913 Devise a way to turn off SLA alerts for bundle/coordinator flexibly
> 
> 
> Diffs
> -
> 
>   client/src/main/java/org/apache/oozie/cli/OozieCLI.java bc8fa50 
>   client/src/main/java/org/apache/oozie/client/OozieClient.java 800e871 
>   
> client/src/main/java/org/apache/oozie/client/event/jms/JMSHeaderConstants.java
>  801ad7e 
>   client/src/main/java/org/apache/oozie/client/rest/JsonTags.java b7cf0e7 
>   client/src/main/java/org/apache/oozie/client/rest/RestConstants.java 
> 85efecf 
>   core/src/main/java/org/apache/oozie/BaseEngine.java a982ad4 
>   core/src/main/java/org/apache/oozie/BundleEngine.java 27afe73 
>   core/src/main/java/org/apache/oozie/CoordinatorActionBean.java bd01d14 
>   core/src/main/java/org/apache/oozie/CoordinatorEngine.java 3406c6f 
>   core/src/main/java/org/apache/oozie/CoordinatorJobBean.java 4d6b970 
>   core/src/main/java/org/apache/oozie/DagEngine.java 70ddd44 
>   core/src/main/java/org/apache/oozie/ErrorCode.java c87 
>   core/src/main/java/org/apache/oozie/command/SlaAlertsXCommand.java e69de29 
>   
> core/src/main/java/org/apache/oozie/command/bundle/BundleSLAAlertsDisableXCommand.java
>  e69de29 
>   
> core/src/main/java/org/apache/oozie/command/bundle/BundleSLAAlertsEnableXCommand.java
>  e69de29 
>   
> core/src/main/java/org/apache/oozie/command/bundle/BundleSLAChangeXCommand.java
>  e69de29 
>   
> core/src/main/java/org/apache/oozie/command/bundle/BundleSlaAlertsXCommand.java
>  e69de29 
>   
> core/src/main/java/org/apache/oozie/command/coord/CoordMaterializeTransitionXCommand.java
>  548946f 
>   
> core/src/main/java/org/apache/oozie/command/coord/CoordSLAAlertsDisableXCommand.java
>  e69de29 
>   
> core/src/main/java/org/apache/oozie/command/coord/CoordSLAAlertsEnableXCommand.java
>  e69de29 
>   
> core/src/main/java/org/apache/oozie/command/coord/CoordSLAChangeXCommand.java 
> e69de29 
>   
> core/src/main/java/org/apache/oozie/command/coord/CoordSlaAlertsXCommand.java 
> e69de29 
>   core/src/main/java/org/apache/oozie/coord/CoordUtils.java 4643d73 
>   
> core/src/main/java/org/apache/oozie/executor/jpa/CoordActionQueryExecutor.java
>  e6ab09b 
>   core/src/main/java/org/apache/oozie/executor/jpa/CoordJobQueryExecutor.java 
> 4bccef4 
>   
> core/src/main/java/org/apache/oozie/executor/jpa/SLARegistrationQueryExecutor.java
>  e220c01 
>   
> core/src/main/java/org/apache/oozie/executor/jpa/SLASummaryQueryExecutor.java 
> c3197b7 
>   
> core/src/main/java/org/apache/oozie/service/Coord

Re: Review Request 28842: OOZIE-1913 Devise a way to turn off SLA alerts for bundle/coordinator flexibly

[Rohini Palaniswamy]

> On Jan. 20, 2015, 11:41 p.m., Rohini Palaniswamy wrote:
> > core/src/test/java/org/apache/oozie/command/TestSlaAlertCommand.java, line 
> > 105
> > <https://reviews.apache.org/r/28842/diff/3/?file=822345#file822345line105>
> >
> > testBundle
> 
> Purshotam Shah wrote:
> This function only test bundle sla change command, so 
> TestBundleSlaChangeCommand  is proper name.

Actually comment was meant as Test->test. Method names should start with 
lowercase.


- Rohini


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28842/#review68798
---


On Jan. 26, 2015, 11:14 p.m., Purshotam Shah wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28842/
> ---
> 
> (Updated Jan. 26, 2015, 11:14 p.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-1913
> https://issues.apache.org/jira/browse/OOZIE-1913
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> OOZIE-1913 Devise a way to turn off SLA alerts for bundle/coordinator flexibly
> 
> 
> Diffs
> -
> 
>   client/src/main/java/org/apache/oozie/cli/OozieCLI.java bc8fa50 
>   client/src/main/java/org/apache/oozie/client/OozieClient.java 800e871 
>   
> client/src/main/java/org/apache/oozie/client/event/jms/JMSHeaderConstants.java
>  801ad7e 
>   client/src/main/java/org/apache/oozie/client/rest/JsonTags.java b7cf0e7 
>   client/src/main/java/org/apache/oozie/client/rest/RestConstants.java 
> 85efecf 
>   core/src/main/java/org/apache/oozie/BaseEngine.java a982ad4 
>   core/src/main/java/org/apache/oozie/BundleEngine.java 27afe73 
>   core/src/main/java/org/apache/oozie/CoordinatorActionBean.java bd01d14 
>   core/src/main/java/org/apache/oozie/CoordinatorEngine.java 3406c6f 
>   core/src/main/java/org/apache/oozie/CoordinatorJobBean.java 4d6b970 
>   core/src/main/java/org/apache/oozie/DagEngine.java 70ddd44 
>   core/src/main/java/org/apache/oozie/ErrorCode.java c87 
>   core/src/main/java/org/apache/oozie/command/SlaAlertsXCommand.java e69de29 
>   
> core/src/main/java/org/apache/oozie/command/bundle/BundleSLAAlertsDisableXCommand.java
>  e69de29 
>   
> core/src/main/java/org/apache/oozie/command/bundle/BundleSLAAlertsEnableXCommand.java
>  e69de29 
>   
> core/src/main/java/org/apache/oozie/command/bundle/BundleSLAChangeXCommand.java
>  e69de29 
>   
> core/src/main/java/org/apache/oozie/command/bundle/BundleSlaAlertsXCommand.java
>  e69de29 
>   
> core/src/main/java/org/apache/oozie/command/coord/CoordMaterializeTransitionXCommand.java
>  548946f 
>   
> core/src/main/java/org/apache/oozie/command/coord/CoordSLAAlertsDisableXCommand.java
>  e69de29 
>   
> core/src/main/java/org/apache/oozie/command/coord/CoordSLAAlertsEnableXCommand.java
>  e69de29 
>   
> core/src/main/java/org/apache/oozie/command/coord/CoordSLAChangeXCommand.java 
> e69de29 
>   
> core/src/main/java/org/apache/oozie/command/coord/CoordSlaAlertsXCommand.java 
> e69de29 
>   core/src/main/java/org/apache/oozie/coord/CoordUtils.java 4643d73 
>   
> core/src/main/java/org/apache/oozie/executor/jpa/CoordActionQueryExecutor.java
>  e6ab09b 
>   core/src/main/java/org/apache/oozie/executor/jpa/CoordJobQueryExecutor.java 
> 4bccef4 
>   
> core/src/main/java/org/apache/oozie/executor/jpa/SLARegistrationQueryExecutor.java
>  e220c01 
>   
> core/src/main/java/org/apache/oozie/executor/jpa/SLASummaryQueryExecutor.java 
> c3197b7 
>   
> core/src/main/java/org/apache/oozie/service/CoordMaterializeTriggerService.java
>  fa16d1d 
>   core/src/main/java/org/apache/oozie/service/EventHandlerService.java 
> 7c0d3be 
>   core/src/main/java/org/apache/oozie/servlet/BaseJobServlet.java c94d1e2 
>   core/src/main/java/org/apache/oozie/servlet/SLAServlet.java 2578e41 
>   core/src/main/java/org/apache/oozie/servlet/V0JobServlet.java b160b46 
>   core/src/main/java/org/apache/oozie/servlet/V1JobServlet.java eed7ca1 
>   core/src/main/java/org/apache/oozie/servlet/V2JobServlet.java da81b49 
>   core/src/main/java/org/apache/oozie/servlet/V2SLAServlet.java a0fe1b6 
>   core/src/main/java/org/apache/oozie/sla/SLACalcStatus.java 189d5ea 
>   core/src/main/java/org/apache/oozie/sla/SLACalculator.java 20f93b5 
>   core/src/main/java/org/apache/oozie/sla/SLACalculatorMemory.java fdce6b5 
>   core/src/main/java/org/apache/oozie/sla/SLAOperations.java f5fc826 
>   core/src/main/java/org/apache/oozie/sla/SLARegistrati

Re: Oozie branching

[Rohini Palaniswamy]

I am not sure. It needs commitment and some good deal of management to keep
a release branch like that for features and tracking what went where and
ensuring things went into proper releases and not missed. Hadoop has
that because they work on some big features which take a long time. What
other projects like Pig, Hive, Tez, etc do is maintain a separate branch
for those and merge them when that work is ready and more stable. For eg:
Pig had separate branches for Tez, Spark, mavenization. Same with Hive. Tez
had separate one for UI. This model enables faster development, but the
responsibility falls on the folks who maintain the branch and develop on
those to stabilize that and ensure proper merge to trunk. We even did that
with Hcat integration for Oozie. With the other model, everyone needs to
ensure they are checking in more than one branch and it gets slightly
complicated. Also it takes away the motivation to keep the trunk in a
highly stable state and would be a more time consuming exercise to make it
stable if we don't stabilize it often enough by branching from it. Just my
two cents bases on my experience with other projects.

Regards,
Rohini

On Fri, Jan 30, 2015 at 5:57 PM, Robert Kanter  wrote:

> Hi all,
>
> Based on some of the recent large changes to the build and some issues
> there, I wonder if it might be a good idea to adopt a branching model like
> Hadoop does.  They have separate "branch-2" and "trunk" branches so that
> things can be committed that should not go into a Hadoop 2.x release can be
> committed to trunk and make it into Hadoop 3.x.  Perhaps we should do
> something similar?
>
>
>- A new "branch-4" that is the dev branch for Oozie 4.x
>- Version 4.2.0-SNAPSHOT
>- When it's time to do Oozie 4.2, we make a new "branch-4.2" based on
>"branch-4" and change the version of "branch-4" to 4.3.0-SNAPSHOT
>- The "master" (trunk) branch can serve as a place for longer term
>commits that would wait for Oozie 5.x
>- Version 5.0.0-SNAPSHOT
>
> The only difference in committing patches is that you'd commit them to
> "master" first, and if it's also meant for Oozie 4, cherry-pick that back
> to "branch-4" too.
>
> Otherwise, we've recently been just taking the current "master" branch and
> making it the next Oozie release, which I'm not sure is the best strategy.
>
> A good candidate for this would be a patch that drops Hadoop 1 and 0.23
> (using only Hadoop 2).  That's probably something for Oozie 5.0, but
> there's nowhere to put that for now.
>
> thoughts?
>
>  - Robert
>

Re: Review Request 28842: OOZIE-1913 Devise a way to turn off SLA alerts for bundle/coordinator flexibly

[Rohini Palaniswamy]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28842/#review72966
---



core/src/main/java/org/apache/oozie/CoordinatorActionBean.java
<https://reviews.apache.org/r/28842/#comment119085>

Why is this removed? Causing test failures in TestPurgeXCommand


https://builds.apache.org/job/oozie-trunk-precommit-build/2223/testReport/junit/org.apache.oozie.command/TestPurgeXCommand/testSucCoordPurgeXCommand/



core/src/main/resources/oozie-default.xml
<https://reviews.apache.org/r/28842/#comment119086>

Unnecessary change. To be reverted



core/src/test/java/org/apache/oozie/command/TestSlaAlertCommand.java
<https://reviews.apache.org/r/28842/#comment119087>

Rename class to TestSLA.


- Rohini Palaniswamy


On Feb. 18, 2015, 6:05 a.m., Purshotam Shah wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28842/
> ---
> 
> (Updated Feb. 18, 2015, 6:05 a.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-1913
> https://issues.apache.org/jira/browse/OOZIE-1913
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> OOZIE-1913 Devise a way to turn off SLA alerts for bundle/coordinator flexibly
> 
> 
> Diffs
> -
> 
>   client/src/main/java/org/apache/oozie/cli/OozieCLI.java 6690869 
>   client/src/main/java/org/apache/oozie/client/OozieClient.java e4c93cd 
>   client/src/main/java/org/apache/oozie/client/rest/JsonTags.java b7cf0e7 
>   client/src/main/java/org/apache/oozie/client/rest/RestConstants.java 
> 3c2afc3 
>   core/src/main/java/org/apache/oozie/BaseEngine.java bf38a0c 
>   core/src/main/java/org/apache/oozie/BundleEngine.java 9818acc 
>   core/src/main/java/org/apache/oozie/CoordinatorActionBean.java bd01d14 
>   core/src/main/java/org/apache/oozie/CoordinatorEngine.java 136c097 
>   core/src/main/java/org/apache/oozie/CoordinatorJobBean.java 4d6b970 
>   core/src/main/java/org/apache/oozie/DagEngine.java 50aef2f 
>   core/src/main/java/org/apache/oozie/ErrorCode.java c87 
>   core/src/main/java/org/apache/oozie/command/SLAAlertsXCommand.java e69de29 
>   
> core/src/main/java/org/apache/oozie/command/bundle/BundleSLAAlertsDisableXCommand.java
>  e69de29 
>   
> core/src/main/java/org/apache/oozie/command/bundle/BundleSLAAlertsEnableXCommand.java
>  e69de29 
>   
> core/src/main/java/org/apache/oozie/command/bundle/BundleSLAChangeXCommand.java
>  e69de29 
>   
> core/src/main/java/org/apache/oozie/command/bundle/BundleSlaAlertsXCommand.java
>  e69de29 
>   
> core/src/main/java/org/apache/oozie/command/bundle/BundleStatusTransitXCommand.java
>  d6a3197 
>   
> core/src/main/java/org/apache/oozie/command/coord/CoordMaterializeTransitionXCommand.java
>  548946f 
>   
> core/src/main/java/org/apache/oozie/command/coord/CoordSLAAlertsDisableXCommand.java
>  e69de29 
>   
> core/src/main/java/org/apache/oozie/command/coord/CoordSLAAlertsEnableXCommand.java
>  e69de29 
>   
> core/src/main/java/org/apache/oozie/command/coord/CoordSLAChangeXCommand.java 
> e69de29 
>   
> core/src/main/java/org/apache/oozie/command/coord/CoordSlaAlertsXCommand.java 
> e69de29 
>   core/src/main/java/org/apache/oozie/coord/CoordUtils.java 4643d73 
>   
> core/src/main/java/org/apache/oozie/executor/jpa/CoordActionQueryExecutor.java
>  e6ab09b 
>   
> core/src/main/java/org/apache/oozie/executor/jpa/CoordJobGetActionIdsForDateRangeJPAExecutor.java
>  1862c7c 
>   
> core/src/main/java/org/apache/oozie/executor/jpa/CoordJobGetActionsByDatesForKillJPAExecutor.java
>  eb95591 
>   
> core/src/main/java/org/apache/oozie/executor/jpa/CoordJobGetActionsForDatesJPAExecutor.java
>  d1856ae 
>   core/src/main/java/org/apache/oozie/executor/jpa/CoordJobQueryExecutor.java 
> 4bccef4 
>   
> core/src/main/java/org/apache/oozie/executor/jpa/CoordJobsToBeMaterializedJPAExecutor.java
>  5e018c7 
>   
> core/src/main/java/org/apache/oozie/executor/jpa/SLARegistrationQueryExecutor.java
>  e220c01 
>   
> core/src/main/java/org/apache/oozie/executor/jpa/SLASummaryQueryExecutor.java 
> c3197b7 
>   
> core/src/main/java/org/apache/oozie/service/CoordMaterializeTriggerService.java
>  fa16d1d 
>   core/src/main/java/org/apache/oozie/service/EventHandlerService.java 
> 7c0d3be 
>   core/src/main/java/org/apache/oozie/servlet/BaseJobServlet.java 5690787 
>   core/src/main/java/org/apache/oozie/servlet/SLAServlet.java 2578e41 
>   core/src/main/java/org/apache/o

Re: Review Request 28842: OOZIE-1913 Devise a way to turn off SLA alerts for bundle/coordinator flexibly

[Rohini Palaniswamy]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28842/#review73008
---

Ship it!


- Rohini Palaniswamy


On Feb. 18, 2015, 8:24 p.m., Purshotam Shah wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28842/
> ---
> 
> (Updated Feb. 18, 2015, 8:24 p.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-1913
> https://issues.apache.org/jira/browse/OOZIE-1913
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> OOZIE-1913 Devise a way to turn off SLA alerts for bundle/coordinator flexibly
> 
> 
> Diffs
> -
> 
>   client/src/main/java/org/apache/oozie/cli/OozieCLI.java 6690869 
>   client/src/main/java/org/apache/oozie/client/OozieClient.java e4c93cd 
>   client/src/main/java/org/apache/oozie/client/event/SLAEvent.java 27a0e1f 
>   client/src/main/java/org/apache/oozie/client/rest/JsonTags.java b7cf0e7 
>   client/src/main/java/org/apache/oozie/client/rest/RestConstants.java 
> 3c2afc3 
>   core/src/main/java/org/apache/oozie/BaseEngine.java bf38a0c 
>   core/src/main/java/org/apache/oozie/BundleEngine.java 9818acc 
>   core/src/main/java/org/apache/oozie/CoordinatorActionBean.java bd01d14 
>   core/src/main/java/org/apache/oozie/CoordinatorEngine.java 136c097 
>   core/src/main/java/org/apache/oozie/CoordinatorJobBean.java 4d6b970 
>   core/src/main/java/org/apache/oozie/DagEngine.java 50aef2f 
>   core/src/main/java/org/apache/oozie/ErrorCode.java c87 
>   core/src/main/java/org/apache/oozie/command/SLAAlertsXCommand.java e69de29 
>   
> core/src/main/java/org/apache/oozie/command/bundle/BundleSLAAlertsDisableXCommand.java
>  e69de29 
>   
> core/src/main/java/org/apache/oozie/command/bundle/BundleSLAAlertsEnableXCommand.java
>  e69de29 
>   
> core/src/main/java/org/apache/oozie/command/bundle/BundleSLAChangeXCommand.java
>  e69de29 
>   
> core/src/main/java/org/apache/oozie/command/bundle/BundleSlaAlertsXCommand.java
>  e69de29 
>   
> core/src/main/java/org/apache/oozie/command/bundle/BundleStatusTransitXCommand.java
>  d6a3197 
>   
> core/src/main/java/org/apache/oozie/command/coord/CoordMaterializeTransitionXCommand.java
>  548946f 
>   
> core/src/main/java/org/apache/oozie/command/coord/CoordSLAAlertsDisableXCommand.java
>  e69de29 
>   
> core/src/main/java/org/apache/oozie/command/coord/CoordSLAAlertsEnableXCommand.java
>  e69de29 
>   
> core/src/main/java/org/apache/oozie/command/coord/CoordSLAAlertsXCommand.java 
> e69de29 
>   
> core/src/main/java/org/apache/oozie/command/coord/CoordSLAChangeXCommand.java 
> e69de29 
>   core/src/main/java/org/apache/oozie/coord/CoordUtils.java 4643d73 
>   
> core/src/main/java/org/apache/oozie/executor/jpa/CoordActionQueryExecutor.java
>  e6ab09b 
>   
> core/src/main/java/org/apache/oozie/executor/jpa/CoordJobGetActionIdsForDateRangeJPAExecutor.java
>  1862c7c 
>   
> core/src/main/java/org/apache/oozie/executor/jpa/CoordJobGetActionsByDatesForKillJPAExecutor.java
>  eb95591 
>   
> core/src/main/java/org/apache/oozie/executor/jpa/CoordJobGetActionsForDatesJPAExecutor.java
>  d1856ae 
>   core/src/main/java/org/apache/oozie/executor/jpa/CoordJobQueryExecutor.java 
> 4bccef4 
>   
> core/src/main/java/org/apache/oozie/executor/jpa/CoordJobsToBeMaterializedJPAExecutor.java
>  5e018c7 
>   
> core/src/main/java/org/apache/oozie/executor/jpa/SLARegistrationQueryExecutor.java
>  e220c01 
>   
> core/src/main/java/org/apache/oozie/executor/jpa/SLASummaryQueryExecutor.java 
> c3197b7 
>   
> core/src/main/java/org/apache/oozie/service/CoordMaterializeTriggerService.java
>  fa16d1d 
>   core/src/main/java/org/apache/oozie/service/EventHandlerService.java 
> 7c0d3be 
>   core/src/main/java/org/apache/oozie/servlet/BaseJobServlet.java 5690787 
>   core/src/main/java/org/apache/oozie/servlet/SLAServlet.java 2578e41 
>   core/src/main/java/org/apache/oozie/servlet/V0JobServlet.java 3e186f9 
>   core/src/main/java/org/apache/oozie/servlet/V1JobServlet.java 64b97c2 
>   core/src/main/java/org/apache/oozie/servlet/V2JobServlet.java 5238426 
>   core/src/main/java/org/apache/oozie/servlet/V2SLAServlet.java a0fe1b6 
>   core/src/main/java/org/apache/oozie/sla/SLACalcStatus.java 189d5ea 
>   core/src/main/java/org/apache/oozie/sla/SLACalculator.java 20f93b5 
>   core/src/main/java/org/apache/oozie/sla/SLACalculatorMemory.java fdce6b5 
>   core/src/main/java/org/apache/oozie/sla/SLAOperations.jav

Re: Review Request 31687: OOZIE-2160 Support attachment in email action

[Rohini Palaniswamy]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31687/#review75495
---



core/src/main/java/org/apache/oozie/action/email/EmailActionExecutor.java
<https://reviews.apache.org/r/31687/#comment122616>

element.getChildTextTrim(ATTACHMENT, ns) -> attachment



core/src/test/java/org/apache/oozie/action/email/TestEmailActionExecutor.java
<https://reviews.apache.org/r/31687/#comment122617>

You should not be allowing local file attachment. I can use that to fetch a 
password file on the Oozie server host and is a security risk.



docs/src/site/twiki/DG_EmailActionExtension.twiki
<https://reviews.apache.org/r/31687/#comment122618>

[COMMA-SEPARATED-HDFS-FILE-PATHS]



docs/src/site/twiki/DG_EmailActionExtension.twiki
<https://reviews.apache.org/r/31687/#comment122619>

Correction of plural usage

The =attachment= is used to attach HDFS file(s) to the mail. Multiple 
attachments can be


- Rohini Palaniswamy


On March 5, 2015, 1:45 p.m., Ryota Egashira wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/31687/
> ---
> 
> (Updated March 5, 2015, 1:45 p.m.)
> 
> 
> Review request for oozie.
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> https://issues.apache.org/jira/browse/OOZIE-2160
> 
> 
> Diffs
> -
> 
>   client/src/main/resources/email-action-0.2.xsd 60ce545 
>   core/src/main/java/org/apache/oozie/action/email/EmailActionExecutor.java 
> 21c6313 
>   
> core/src/main/java/org/apache/oozie/service/AbandonedCoordCheckerService.java 
> ec8cf71 
>   
> core/src/test/java/org/apache/oozie/action/email/TestEmailActionExecutor.java 
> e1f314e 
>   docs/src/site/twiki/DG_EmailActionExtension.twiki 695e16b 
> 
> Diff: https://reviews.apache.org/r/31687/diff/
> 
> 
> Testing
> ---
> 
> test done. tested in QE enviornment with real HDFS
> 
> 
> Thanks,
> 
> Ryota Egashira
> 
>

Re: Review Request 31687: OOZIE-2160 Support attachment in email action

[Rohini Palaniswamy]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31687/#review75505
---



core/src/main/java/org/apache/oozie/action/email/EmailActionExecutor.java
<https://reviews.apache.org/r/31687/#comment122628>

You will have to do doAs as that user to access the hdfs file. Oozie user 
will not have permissions to read it.


- Rohini Palaniswamy


On March 5, 2015, 1:45 p.m., Ryota Egashira wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/31687/
> ---
> 
> (Updated March 5, 2015, 1:45 p.m.)
> 
> 
> Review request for oozie.
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> https://issues.apache.org/jira/browse/OOZIE-2160
> 
> 
> Diffs
> -
> 
>   client/src/main/resources/email-action-0.2.xsd 60ce545 
>   core/src/main/java/org/apache/oozie/action/email/EmailActionExecutor.java 
> 21c6313 
>   
> core/src/main/java/org/apache/oozie/service/AbandonedCoordCheckerService.java 
> ec8cf71 
>   
> core/src/test/java/org/apache/oozie/action/email/TestEmailActionExecutor.java 
> e1f314e 
>   docs/src/site/twiki/DG_EmailActionExtension.twiki 695e16b 
> 
> Diff: https://reviews.apache.org/r/31687/diff/
> 
> 
> Testing
> ---
> 
> test done. tested in QE enviornment with real HDFS
> 
> 
> Thanks,
> 
> Ryota Egashira
> 
>

Re: Review Request 31232: OOZIE-2146 add SLA bulk API

[Rohini Palaniswamy]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31232/#review75534
---



core/src/main/java/org/apache/oozie/executor/jpa/sla/SLASummaryGetForFilterJPAExecutor.java
<https://reviews.apache.org/r/31232/#comment122673>

If one of jobId, parentId, bundle, nominal_time filters is not specified 
throw an error asking to specify them.



core/src/main/java/org/apache/oozie/executor/jpa/sla/SLASummaryGetForFilterJPAExecutor.java
<https://reviews.apache.org/r/31232/#comment122704>

Do debug logging of the constructed query.



core/src/main/java/org/apache/oozie/executor/jpa/sla/SLASummaryGetForFilterJPAExecutor.java
<https://reviews.apache.org/r/31232/#comment122675>

Can't we use the eventProcessed bit? Query will be simpler



core/src/main/java/org/apache/oozie/executor/jpa/sla/SLASummaryGetForFilterJPAExecutor.java
<https://reviews.apache.org/r/31232/#comment122701>

CURRENT_TIMESTAMP - Isn't this database specific? We should probably add it 
as query param



core/src/main/java/org/apache/oozie/executor/jpa/sla/SLASummaryGetForFilterJPAExecutor.java
<https://reviews.apache.org/r/31232/#comment122702>

Can directly put -1 here. Also expectedDuration != -1 is enough.



core/src/main/java/org/apache/oozie/servlet/V2SLAServlet.java
<https://reviews.apache.org/r/31232/#comment122677>

Let us just do .* instead of Services.get().getSystemId(). If that changes 
older bundles will not be matched



core/src/main/java/org/apache/oozie/servlet/V2SLAServlet.java
<https://reviews.apache.org/r/31232/#comment122679>

Get rid of the bulk and return all the new fields everytime.



core/src/test/java/org/apache/oozie/servlet/TestV2SLAServlet.java
<https://reviews.apache.org/r/31232/#comment122680>

?



docs/src/site/twiki/DG_SLAMonitoring.twiki
<https://reviews.apache.org/r/31232/#comment122682>

Remove this line



docs/src/site/twiki/DG_SLAMonitoring.twiki
<https://reviews.apache.org/r/31232/#comment122683>

Bundle Job ID or Bundle App Name. Fetches SLA information for actions of 
all coordinators in that bundle.



docs/src/site/twiki/DG_SLAMonitoring.twiki
<https://reviews.apache.org/r/31232/#comment122690>

with comma separation. When multiple statuses are specified, it is 
considered as OR.



docs/src/site/twiki/DG_SLAMonitoring.twiki
<https://reviews.apache.org/r/31232/#comment122684>

coordiantor -> coordinator



docs/src/site/twiki/DG_SLAMonitoring.twiki
<https://reviews.apache.org/r/31232/#comment122685>

Remove this example and just add new fields to old examples.

If you are keeping it change title to "All Coordinator actions in a Bundle"



docs/src/site/twiki/DG_SLAMonitoring.twiki
<https://reviews.apache.org/r/31232/#comment122688>

Should be number



docs/src/site/twiki/DG_SLAMonitoring.twiki
<https://reviews.apache.org/r/31232/#comment122689>

Remove this line. The filters should apply in all cases.



docs/src/site/twiki/DG_SLAMonitoring.twiki
<https://reviews.apache.org/r/31232/#comment122686>

Return negative value even for met.


- Rohini Palaniswamy


On Feb. 25, 2015, 7:44 p.m., Ryota Egashira wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/31232/
> ---
> 
> (Updated Feb. 25, 2015, 7:44 p.m.)
> 
> 
> Review request for oozie.
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> https://issues.apache.org/jira/browse/OOZIE-2146
> 
> 
> Diffs
> -
> 
>   client/src/main/java/org/apache/oozie/client/OozieClient.java 5de25cc 
>   client/src/main/java/org/apache/oozie/client/rest/JsonTags.java 1022dd7 
>   
> core/src/main/java/org/apache/oozie/executor/jpa/sla/SLASummaryGetForFilterJPAExecutor.java
>  b55cda3 
>   core/src/main/java/org/apache/oozie/servlet/V2SLAServlet.java 57170e1 
>   core/src/main/java/org/apache/oozie/sla/SLASummaryBean.java a88dcf6 
>   core/src/test/java/org/apache/oozie/servlet/DagServletTestCase.java 48193c7 
>   core/src/test/java/org/apache/oozie/servlet/TestV2SLAServlet.java 1886f48 
>   docs/src/site/twiki/DG_SLAMonitoring.twiki 1413945 
> 
> Diff: https://reviews.apache.org/r/31232/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Ryota Egashira
> 
>

Re: Review Request 31687: OOZIE-2160 Support attachment in email action

[Rohini Palaniswamy]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31687/#review75764
---



core/src/main/java/org/apache/oozie/action/email/EmailActionExecutor.java
<https://reviews.apache.org/r/31687/#comment123021>

LOG.warn


- Rohini Palaniswamy


On March 7, 2015, 5:54 a.m., Ryota Egashira wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/31687/
> ---
> 
> (Updated March 7, 2015, 5:54 a.m.)
> 
> 
> Review request for oozie.
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> https://issues.apache.org/jira/browse/OOZIE-2160
> 
> 
> Diffs
> -
> 
>   client/src/main/resources/email-action-0.2.xsd 60ce545 
>   core/src/main/java/org/apache/oozie/action/email/EmailActionExecutor.java 
> 21c6313 
>   
> core/src/main/java/org/apache/oozie/service/AbandonedCoordCheckerService.java 
> ec8cf71 
>   
> core/src/test/java/org/apache/oozie/action/email/TestEmailActionExecutor.java 
> e1f314e 
>   docs/src/site/twiki/DG_EmailActionExtension.twiki 695e16b 
> 
> Diff: https://reviews.apache.org/r/31687/diff/
> 
> 
> Testing
> ---
> 
> test done. tested in QE enviornment with real HDFS
> 
> 
> Thanks,
> 
> Ryota Egashira
> 
>

Re: Review Request 31687: OOZIE-2160 Support attachment in email action

[Rohini Palaniswamy]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31687/#review75776
---

Ship it!


Ship It!

- Rohini Palaniswamy


On March 9, 2015, 8:45 p.m., Ryota Egashira wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/31687/
> ---
> 
> (Updated March 9, 2015, 8:45 p.m.)
> 
> 
> Review request for oozie.
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> https://issues.apache.org/jira/browse/OOZIE-2160
> 
> 
> Diffs
> -
> 
>   client/src/main/resources/email-action-0.2.xsd 60ce545 
>   core/src/main/java/org/apache/oozie/action/email/EmailActionExecutor.java 
> 21c6313 
>   
> core/src/main/java/org/apache/oozie/service/AbandonedCoordCheckerService.java 
> ec8cf71 
>   
> core/src/test/java/org/apache/oozie/action/email/TestEmailActionExecutor.java 
> e1f314e 
>   docs/src/site/twiki/DG_EmailActionExtension.twiki 695e16b 
> 
> Diff: https://reviews.apache.org/r/31687/diff/
> 
> 
> Testing
> ---
> 
> test done. tested in QE enviornment with real HDFS
> 
> 
> Thanks,
> 
> Ryota Egashira
> 
>

Re: [DISCUSSION] Oozie 4.2.0 release

[Rohini Palaniswamy]

We don't have a list of issues that need to be included. But Michelle ran
the regression with trunk yesterday and there were couple of failures. We
haven't investigated them yet. Those will have to be fixed though. Ryota
will file jiras and upload fixes next week after investigating them.

Regards,
Rohini

On Thu, Mar 26, 2015 at 1:58 PM, Harsh J  wrote:

> I can take over OOZIE-1963. I need to test the modified hive(1) example and
> will then attach the change to the JIRA.
>
> On Thu, Mar 26, 2015 at 6:20 AM, Robert Kanter 
> wrote:
>
> > I was hoping one of our QA engineers would take care of that, but it
> looks
> > like he won't have time.  I don't think I'm proficient enough in Hive
> > Server 2 to make the example, so I've unassigned OOZIE-1963 from me.
> > Though I think it would still be good to include.
> >
> > If we include the Spark action, I think we should also include
> OOZIE-2170.
> > OOZIE-2174 would also be good; it addresses some long-standing
> shortcomings
> > of OozieClient/OozieCLI.
> > Finally, we should also include OOZIE-2181, which fixes some problems
> > retrieving job info.
> > I have patches available for all 3 of these.
> >
> >
> > - Robert
> >
> > On Wed, Mar 25, 2015 at 10:58 AM, Shwetha Shivalingamurthy <
> > sshivalingamur...@hortonworks.com> wrote:
> >
> > > Robert, will you be able to take up OOZIE-1963?
> > >
> > > Dev team, Are there any other un-resolved jiras that needs to be part
> of
> > > 4.2 release?
> > >
> > > Thanks,
> > > Shwetha
> > >
> > >
> > > On 11/02/15 1:44 pm, "Robert Kanter"  wrote:
> > >
> > > >I 4.2.0 release sounds good; I don't have cycles to drive it though.
> > > >
> > > >The Spark action currently has a number of problems with it.  I'd hold
> > off
> > > >on putting that into a release until we get those fixed (OOZIE-2071
> > > > (it's to add an
> > > >example,
> > > >but the comments discuss the problems), OOZIE-2087
> > > >).
> > > >
> > > >I'm all for including the Hive2 action; though it would be good if we
> > > >could
> > > >create an example workflow for it for completeness (OOZIE-1963
> > > >).  I can probably
> > take
> > > >care of this relatively easily.
> > > >
> > > >
> > > >- Robert
> > > >
> > > >On Tue, Feb 10, 2015 at 9:56 PM, Shwetha Shivalingamurthy <
> > > >sshivalingamur...@hortonworks.com> wrote:
> > > >
> > > >> Hi All,
> > > >>
> > > >> Oozie trunk has a couple of changes since the last oozie release -
> > > >>re-run
> > > >> fixes, config cleanup, spark action, hive2 action, external
> > > >>configuration
> > > >> for mapred action, better error reporting and many other bug fixes.
> > > >>
> > > >> Can we have a oozie release from trunk? If someone wants to
> volunteer
> > to
> > > >> drive the release, please take this forward. Else, I can drive it as
> > > >>well.
> > > >>
> > > >> If we want to do a release, let's identify the pending jiras which
> are
> > > >> required for this release.
> > > >>
> > > >> Thanks,
> > > >> Shwetha
> > > >>
> > >
> > >
> >
>
>
>
> --
> Harsh J
>

Re: [DISCUSS] get rid of release-log.txt?

[Rohini Palaniswamy]

I agree with Bowen. That is the first place we look for changes that have
gone in. Running queries on JIRA or going through git logs is cumbersome
and not easy to go over multiple releases. Also currently Fix version is
not followed properly at all. We are more disciplined with release-log.txt.

Regards,
Rohini

On Thu, Mar 26, 2015 at 12:56 PM, bowen zhang <
bowenzhang...@yahoo.com.invalid> wrote:

> I don't see a good alternative to track which jira goes into which
> branch.I would advise against removing release-log.txt
>
>   From: Robert Kanter 
>  To: "dev@oozie.apache.org" 
>  Sent: Thursday, March 26, 2015 12:07 PM
>  Subject: [DISCUSS] get rid of release-log.txt?
>
> Hi all,
>
> Do we need to keep maintaining the release-log.txt?  It's an extra step for
> committing (that we sometimes mess up or forget) and I'm not sure how much
> benefit we get from using it given that we have the same information in
> JIRA and the git log.  Personally, it makes backporting patches to CDH more
> cumbersome because there's almost always a conflict in release-log.txt, so
> I pretty much ignore it.
>
> Hadoop has been discussing getting rid of their CHANGES.txt recently.  And
> while most people seem to agree it would be good to get rid of, they're
> having a lot of discussion on what to use instead.  I think our project is
> small enough compared to Hadoop that git and JIRA should be fine.
>
> What do others think?
>
>
> thanks
> - Robert
>
>
>
>

Re: Review Request 31232: OOZIE-2146 add SLA bulk API

[Rohini Palaniswamy]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31232/#review78177
---



core/src/main/java/org/apache/oozie/executor/jpa/sla/SLASummaryGetForFilterJPAExecutor.java
<https://reviews.apache.org/r/31232/#comment126689>

Remove import



core/src/main/java/org/apache/oozie/executor/jpa/sla/SLASummaryGetForFilterJPAExecutor.java
<https://reviews.apache.org/r/31232/#comment126686>

private



core/src/main/java/org/apache/oozie/executor/jpa/sla/SLASummaryGetForFilterJPAExecutor.java
<https://reviews.apache.org/r/31232/#comment126690>

LOG.debug



docs/src/site/twiki/DG_SLAMonitoring.twiki
<https://reviews.apache.org/r/31232/#comment126691>

No quotes



docs/src/site/twiki/DG_SLAMonitoring.twiki
<https://reviews.apache.org/r/31232/#comment126692>

startDelay/durationDelay/endDelay values returned indicate how much delay 
  (Remove Bulk API)

startDelay/durationDelay/endDelay and eventStatus also need to be added to 
previous examples.



core/src/main/java/org/apache/oozie/executor/jpa/CoordinatorJobGetForUserAppnameJPAExecutor.java
<https://reviews.apache.org/r/31232/#comment126688>

private



core/src/main/java/org/apache/oozie/sla/SLASummaryBean.java
<https://reviews.apache.org/r/31232/#comment126694>

Remove boolean bulk


- Rohini Palaniswamy


On March 23, 2015, 9:25 p.m., Ryota Egashira wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/31232/
> ---
> 
> (Updated March 23, 2015, 9:25 p.m.)
> 
> 
> Review request for oozie.
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> https://issues.apache.org/jira/browse/OOZIE-2146
> 
> 
> Diffs
> -
> 
>   client/src/main/java/org/apache/oozie/client/OozieClient.java 5de25cc 
>   client/src/main/java/org/apache/oozie/client/rest/JsonTags.java 1022dd7 
>   
> core/src/main/java/org/apache/oozie/executor/jpa/CoordinatorJobGetForUserAppnameJPAExecutor.java
>  3bb2f88 
>   
> core/src/main/java/org/apache/oozie/executor/jpa/sla/SLASummaryGetForFilterJPAExecutor.java
>  b55cda3 
>   core/src/main/java/org/apache/oozie/servlet/V2SLAServlet.java 57170e1 
>   core/src/main/java/org/apache/oozie/sla/SLASummaryBean.java a88dcf6 
>   core/src/test/java/org/apache/oozie/servlet/DagServletTestCase.java 48193c7 
>   core/src/test/java/org/apache/oozie/servlet/TestV2SLAServlet.java 1886f48 
>   docs/src/site/twiki/DG_SLAMonitoring.twiki 1413945 
> 
> Diff: https://reviews.apache.org/r/31232/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Ryota Egashira
> 
>

Re: Review Request 32443: provide an option to disable cleanup for output dataset in rerun

[Rohini Palaniswamy]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/32443/#review78179
---



core/src/test/java/org/apache/oozie/command/coord/TestCoordRerunXCommand.java
<https://reviews.apache.org/r/32443/#comment126695>

Can you compare the last modified time of the success file before and after 
rerun?


- Rohini Palaniswamy


On March 24, 2015, 4:25 p.m., Ryota Egashira wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/32443/
> ---
> 
> (Updated March 24, 2015, 4:25 p.m.)
> 
> 
> Review request for oozie.
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> https://issues.apache.org/jira/browse/OOZIE-2183
> 
> 
> Diffs
> -
> 
>   client/src/main/resources/oozie-coordinator-0.4.xsd 78205f6 
>   core/src/main/java/org/apache/oozie/command/coord/CoordRerunXCommand.java 
> 050662e 
>   
> core/src/test/java/org/apache/oozie/command/coord/TestCoordRerunXCommand.java 
> 45457d1 
>   docs/src/site/twiki/CoordinatorFunctionalSpec.twiki 0812bb8 
>   docs/src/site/twiki/DG_CoordinatorRerun.twiki c937da5 
> 
> Diff: https://reviews.apache.org/r/32443/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Ryota Egashira
> 
>

Re: Review Request 32443: OOZIE-2183 provide an option to disable cleanup for output dataset in rerun

[Rohini Palaniswamy]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/32443/#review78535
---

Ship it!


Ship It!

- Rohini Palaniswamy


On March 31, 2015, 11:36 p.m., Ryota Egashira wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/32443/
> ---
> 
> (Updated March 31, 2015, 11:36 p.m.)
> 
> 
> Review request for oozie.
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> https://issues.apache.org/jira/browse/OOZIE-2183
> 
> 
> Diffs
> -
> 
>   client/src/main/resources/oozie-coordinator-0.4.xsd 78205f6 
>   core/src/main/java/org/apache/oozie/command/coord/CoordRerunXCommand.java 
> 050662e 
>   
> core/src/test/java/org/apache/oozie/command/coord/TestCoordRerunXCommand.java 
> 45457d1 
>   core/src/test/resources/coord-rerun-action4.xml PRE-CREATION 
>   docs/src/site/twiki/CoordinatorFunctionalSpec.twiki 0812bb8 
>   docs/src/site/twiki/DG_CoordinatorRerun.twiki c937da5 
> 
> Diff: https://reviews.apache.org/r/32443/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Ryota Egashira
> 
>

Re: Review Request 31232: OOZIE-2146 add SLA bulk API

[Rohini Palaniswamy]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31232/#review78536
---

Ship it!


Ship It!

- Rohini Palaniswamy


On April 1, 2015, 1:42 p.m., Ryota Egashira wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/31232/
> ---
> 
> (Updated April 1, 2015, 1:42 p.m.)
> 
> 
> Review request for oozie.
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> https://issues.apache.org/jira/browse/OOZIE-2146
> 
> 
> Diffs
> -
> 
>   client/src/main/java/org/apache/oozie/client/OozieClient.java 5de25cc 
>   client/src/main/java/org/apache/oozie/client/rest/JsonTags.java 1022dd7 
>   
> core/src/main/java/org/apache/oozie/executor/jpa/CoordinatorJobGetForUserAppnameJPAExecutor.java
>  3bb2f88 
>   
> core/src/main/java/org/apache/oozie/executor/jpa/sla/SLASummaryGetForFilterJPAExecutor.java
>  b55cda3 
>   core/src/main/java/org/apache/oozie/servlet/V2SLAServlet.java 57170e1 
>   core/src/main/java/org/apache/oozie/sla/SLASummaryBean.java a88dcf6 
>   core/src/test/java/org/apache/oozie/servlet/DagServletTestCase.java 48193c7 
>   core/src/test/java/org/apache/oozie/servlet/TestV2SLAServlet.java 1886f48 
>   docs/src/site/twiki/DG_SLAMonitoring.twiki 1413945 
> 
> Diff: https://reviews.apache.org/r/31232/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Ryota Egashira
> 
>

Re: Review Request 33221: OOZIE-2205 add option to load default/site.xml to actionConf on compute node

[Rohini Palaniswamy]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/33221/#review80380
---



core/src/main/java/org/apache/oozie/action/hadoop/Hive2ActionExecutor.java
<https://reviews.apache.org/r/33221/#comment130268>

I am not sure what is the behaviour as script execution is on Hive server 2 
and removing them will not impact Hive server execution, but might impact 
connecting to hive server 2. Unless you can setup one and test the patch, 
please remove the code for hive server 2 action.



core/src/main/java/org/apache/oozie/action/hadoop/HiveActionExecutor.java
<https://reviews.apache.org/r/33221/#comment130262>

loadHadoopDefaultResources



core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java
<https://reviews.apache.org/r/33221/#comment130257>

loadDefaultResources



core/src/main/java/org/apache/oozie/service/HadoopAccessorService.java
<https://reviews.apache.org/r/33221/#comment130258>

ACTION_CONFS_LOAD_DEFAULT_RESOURCES = ACTION_CONFS + 
".load.default.resources";



core/src/main/resources/oozie-default.xml
<https://reviews.apache.org/r/33221/#comment130260>

true means that default and site xml files of hadoop (core-default, 
core-site, hdfs-default, hdfs-site, mapred-default, mapred-site, yarn-default, 
yarn-site) 

false means that site xml files are not loaded on server, instead loaded on 
launcher node. This is only done for pig and hive actions which handle loading 
those files automatically from the classpath on launcher task.

Please capitalize 
  - Oozie server
  - It defaults to



sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveActionExecutor.java
<https://reviews.apache.org/r/33221/#comment130263>

testActionConfLoadDefaultResources . Same in pig test.



sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveActionExecutor.java
<https://reviews.apache.org/r/33221/#comment130266>

a is not used anywhere


- Rohini Palaniswamy


On April 15, 2015, 3:02 p.m., Ryota Egashira wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/33221/
> ---
> 
> (Updated April 15, 2015, 3:02 p.m.)
> 
> 
> Review request for oozie.
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> https://issues.apache.org/jira/browse/OOZIE-2205
> 
> 
> Diffs
> -
> 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2ActionExecutor.java 
> 704b762 
>   core/src/main/java/org/apache/oozie/action/hadoop/HiveActionExecutor.java 
> dbb50ba 
>   core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 
> cf22ed6 
>   core/src/main/java/org/apache/oozie/action/hadoop/PigActionExecutor.java 
> 4d6a1b4 
>   core/src/main/java/org/apache/oozie/service/HadoopAccessorService.java 
> 18de48a 
>   core/src/main/resources/oozie-default.xml 3936fca 
>   
> sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveActionExecutor.java
>  c28839c 
>   
> sharelib/hive2/src/test/java/org/apache/oozie/action/hadoop/TestHive2ActionExecutor.java
>  16d0267 
>   
> sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigActionExecutor.java
>  039bd84 
> 
> Diff: https://reviews.apache.org/r/33221/diff/
> 
> 
> Testing
> ---
> 
> tested on real cluster.
> 
> 
> Thanks,
> 
> Ryota Egashira
> 
>

Re: Review Request 33221: OOZIE-2205 add option to load default/site.xml to actionConf on compute node

[Rohini Palaniswamy]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/33221/#review80524
---

Ship it!


Please do the minor variable renames before checking in.


core/src/main/java/org/apache/oozie/action/hadoop/HiveActionExecutor.java
<https://reviews.apache.org/r/33221/#comment130501>

Can you rename loadOnTask to loadDefaultResources and ACTION_CONFS_LOAD to 
ACTION_CONFS_LOAD_DEFAULT_RESOURCES?



core/src/main/java/org/apache/oozie/action/hadoop/PigActionExecutor.java
<https://reviews.apache.org/r/33221/#comment130500>

Can you rename loadOnTask to loadDefaultResources and ACTION_CONFS_LOAD to 
ACTION_CONFS_LOAD_DEFAULT_RESOURCES?


- Rohini Palaniswamy


On April 17, 2015, 5:56 p.m., Ryota Egashira wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/33221/
> ---
> 
> (Updated April 17, 2015, 5:56 p.m.)
> 
> 
> Review request for oozie.
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> https://issues.apache.org/jira/browse/OOZIE-2205
> 
> 
> Diffs
> -
> 
>   core/src/main/java/org/apache/oozie/action/hadoop/HiveActionExecutor.java 
> dbb50ba 
>   core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 
> cf22ed6 
>   core/src/main/java/org/apache/oozie/action/hadoop/PigActionExecutor.java 
> 4d6a1b4 
>   core/src/main/java/org/apache/oozie/service/HadoopAccessorService.java 
> 18de48a 
>   core/src/main/resources/oozie-default.xml 3936fca 
>   
> sharelib/hive/src/test/java/org/apache/oozie/action/hadoop/TestHiveActionExecutor.java
>  c28839c 
>   
> sharelib/pig/src/test/java/org/apache/oozie/action/hadoop/TestPigActionExecutor.java
>  039bd84 
> 
> Diff: https://reviews.apache.org/r/33221/diff/
> 
> 
> Testing
> ---
> 
> tested on real cluster.
> 
> 
> Thanks,
> 
> Ryota Egashira
> 
>

Re: Review Request 33400: OOZIE-2215 Support glob in FS EL function

[Rohini Palaniswamy]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/33400/#review81348
---



core/src/main/java/org/apache/oozie/action/hadoop/FsELFunctions.java
<https://reviews.apache.org/r/33400/#comment131694>

Please get rid of this check. You have already done the globbing. To throw 
an error after that is meaningless.  NN and DistributedFileSystem are already 
built to handle this by fetching in batches. Need to remove this check 
everywhere - https://issues.apache.org/jira/browse/OOZIE-1682 .



core/src/main/java/org/apache/oozie/action/hadoop/FsELFunctions.java
<https://reviews.apache.org/r/33400/#comment131720>

Redundant code. If globbing returns a list of paths, then it means 
something exists. All you need is

FileStatus[] fileStatuses = fs.globStatus(path);

return (fileStatuses != null and fileStatuses.length > 0);



docs/src/site/twiki/WorkflowFunctionalSpec.twiki
<https://reviews.apache.org/r/33400/#comment131721>

Only need below in documentation

It returns =true= or =false= depending if the specified path URI exists or 
not. If the URI is a glob pattern, it returns true if there is atleast one 
matching path.


- Rohini Palaniswamy


On April 21, 2015, 5:17 p.m., Ryota Egashira wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/33400/
> ---
> 
> (Updated April 21, 2015, 5:17 p.m.)
> 
> 
> Review request for oozie.
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> https://issues.apache.org/jira/browse/OOZIE-2215
> 
> 
> Diffs
> -
> 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsELFunctions.java 
> f8af61c 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestFsELFunctions.java 
> c81fc61 
>   docs/src/site/twiki/WorkflowFunctionalSpec.twiki e3790a4 
> 
> Diff: https://reviews.apache.org/r/33400/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Ryota Egashira
> 
>

Re: Review Request 33400: OOZIE-2215 Support glob in FS EL function

[Rohini Palaniswamy]

> On April 23, 2015, 7:34 p.m., Rohini Palaniswamy wrote:
> > core/src/main/java/org/apache/oozie/action/hadoop/FsELFunctions.java, line 
> > 81
> > <https://reviews.apache.org/r/33400/diff/1/?file=938254#file938254line81>
> >
> > Please get rid of this check. You have already done the globbing. To 
> > throw an error after that is meaningless.  NN and DistributedFileSystem are 
> > already built to handle this by fetching in batches. Need to remove this 
> > check everywhere - https://issues.apache.org/jira/browse/OOZIE-1682 .

Actually more efficient one is to go with the PathFilter suggestion by Puru. 
Throw some specific exception in the accept() method and catch the exception 
and return true. You don't have to do any limit check. Even if you get one Path 
that means it exists and you need to return ture. If it did not go into catch 
block, you return false.

This would minimize the calls to namenode to one listStatus call for first 
pattern. It would be good to avoid the multiple batch calls fs.listStatus 
(DistributedFileSystem.listStatusInternal) makes to NN in case of big list of 
paths, but that is not possible. Filed 
https://issues.apache.org/jira/browse/HDFS-8234 to fix it in hdfs code.


- Rohini


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/33400/#review81348
---


On April 21, 2015, 5:17 p.m., Ryota Egashira wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/33400/
> ---
> 
> (Updated April 21, 2015, 5:17 p.m.)
> 
> 
> Review request for oozie.
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> https://issues.apache.org/jira/browse/OOZIE-2215
> 
> 
> Diffs
> -
> 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsELFunctions.java 
> f8af61c 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestFsELFunctions.java 
> c81fc61 
>   docs/src/site/twiki/WorkflowFunctionalSpec.twiki e3790a4 
> 
> Diff: https://reviews.apache.org/r/33400/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Ryota Egashira
> 
>

Re: Review Request 33948: OOZIE-2229 oozie-hadoop-utils version does not conform to standard

[Rohini Palaniswamy]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/33948/#review82856
---



hadooplibs/hadoop-auth-1/pom.xml
<https://reviews.apache.org/r/33948/#comment133693>

Check if you can have a jar.classifier property which defaults to "" 
specified in parent pom.xml and its maven-jar-plugin definition and just 
override that property value to hadooplib.classifier here so that you don't 
have to define the whole plugin section everytime?



pom.xml
<https://reviews.apache.org/r/33948/#comment133692>

Add  after . Standard definition order is 
groupId,artifactId,version,classfier.


- Rohini Palaniswamy


On May 7, 2015, 5:51 p.m., Ryota Egashira wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/33948/
> ---
> 
> (Updated May 7, 2015, 5:51 p.m.)
> 
> 
> Review request for oozie.
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> WIP, still rough and not fully functional, sharing for early review.
> 
> mvn compile works for hadoop-1, hadoop-2, hadoop-0.23
> mvn package works for hadoop-1, but not for hadoop-2 and 23.
> 
> also org.apache.maven.plugins definition added to all version of poms 
> repeatedly, wonder how to remove redundancy, but  without this doens't work
> 
> 
> Diffs
> -
> 
>   client/pom.xml 73f7ef5 
>   core/pom.xml c62556c 
>   hadooplibs/hadoop-auth-0.23/pom.xml 5769f66 
>   hadooplibs/hadoop-auth-1/pom.xml ac36798 
>   hadooplibs/hadoop-auth-2/pom.xml 977d49c 
>   hadooplibs/hadoop-distcp-0.23/pom.xml c13b78d 
>   hadooplibs/hadoop-distcp-1/pom.xml 5b4f96f 
>   hadooplibs/hadoop-distcp-2/pom.xml 4097cc4 
>   hadooplibs/hadoop-distcp-3/pom.xml 8d8180d 
>   hadooplibs/hadoop-utils-0.23/pom.xml 1bd243e 
>   hadooplibs/hadoop-utils-1/pom.xml 48d97b0 
>   hadooplibs/hadoop-utils-2/pom.xml 832cf5b 
>   hadooplibs/hadoop-utils-3/pom.xml e7df53b 
>   pom.xml ea9ccef 
>   sharelib/distcp/pom.xml a329b7b 
>   sharelib/hive/pom.xml 3da0bdc 
>   sharelib/hive2/pom.xml 41eade9 
>   sharelib/oozie/pom.xml 484fb45 
>   sharelib/pig/pom.xml fd3cd66 
>   sharelib/sqoop/pom.xml 5308bd5 
> 
> Diff: https://reviews.apache.org/r/33948/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Ryota Egashira
> 
>

Re: Review Request 33639: OOZIE-2219 Kill pig child jobs in ActionKillXCommand when sequence file created

[Rohini Palaniswamy]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/33639/#review83276
---



core/src/main/java/org/apache/oozie/action/hadoop/PigActionExecutor.java
<https://reviews.apache.org/r/33639/#comment134179>

This code needs to be called in ActionCheckXCommand as well if the launcher 
job status is killed.



core/src/main/java/org/apache/oozie/action/hadoop/PigActionExecutor.java
<https://reviews.apache.org/r/33639/#comment134177>

The information from the sequence file needs to be updated into the 
database.



core/src/main/java/org/apache/oozie/action/hadoop/PigActionExecutor.java
<https://reviews.apache.org/r/33639/#comment134176>

Have the whole code inside the loop in a try catch block, log the exception 
and continue. If it is already killed or succeeded, killjob() will throw an 
exception



core/src/test/java/org/apache/oozie/command/wf/TestWorkflowActionKillXCommand.java
<https://reviews.apache.org/r/33639/#comment134232>

Remove this line



core/src/test/java/org/apache/oozie/command/wf/TestWorkflowActionKillXCommand.java
<https://reviews.apache.org/r/33639/#comment134233>

Remove this try catch block. catch block is unnecessary


- Rohini Palaniswamy


On April 28, 2015, 8:26 p.m., Ryota Egashira wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/33639/
> ---
> 
> (Updated April 28, 2015, 8:26 p.m.)
> 
> 
> Review request for oozie.
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> https://issues.apache.org/jira/browse/OOZIE-2219
> 
> 
> Diffs
> -
> 
>   core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 
> eb2dbdb 
>   core/src/main/java/org/apache/oozie/action/hadoop/PigActionExecutor.java 
> 8b2dc16 
>   
> core/src/test/java/org/apache/oozie/command/wf/TestWorkflowActionKillXCommand.java
>  72f0114 
> 
> Diff: https://reviews.apache.org/r/33639/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Ryota Egashira
> 
>

Re: [VOTE] Release Oozie 4.2.0 (candidate 0)

[Rohini Palaniswamy]

+1. Checked signatures. Compiled and ran unit tests.

Regards,
Rohini

On Wed, May 27, 2015 at 2:16 AM, Harsh J  wrote:

> +1. I checked the MD5 and the RAT report, and built it
> successfully with the -Phadoop-2 flag.
>
> On Wed, May 27, 2015 at 10:09 AM, Shwetha Shivalingamurthy
>  wrote:
> > Thanks Robert for verifying.
> >
> > I didn¹t realise it was a long weekend in the US. Extending the vote by 2
> > days. Vote now closes on Friday, May 29th, 2015 12:00 PM GMT. Please
> > verify and vote.
> >
> > +1 from me.
> >
> > Thanks,
> > Shwetha
> >
> > On 27/05/15 3:15 am, "Robert Kanter"  wrote:
> >
> >>+1
> >>
> >>
> >>+ Verified md5
> >>+ RAT report looks good
> >>+ Ran some of the examples against Hadoop 1.2.0 after compiling with the
> >>hadoop-1 and uber profiles  (mvn clean package assembly:single
> -DskipTests
> >>-Phadoop-1 -Puber -DgenerateDocs)
> >>+ Ran some of the examples against the latest Hadoop 2.8-SNAPSHOT after
> >>compiling with the hadoop-2  and uber profiles (mvn clean package
> >>assembly:single -DskipTests -Phadoop-2 -Puber -DgenerateDocs)
> >>
> >>- Robert
> >>
> >>
> >>On Tue, May 26, 2015 at 8:04 AM, pavan kumar Kolamuri <
> >>pavan.kolam...@gmail.com> wrote:
> >>
> >>> Hi shwetha  there is issue regarding spark action
> >>> https://issues.apache.org/jira/browse/OOZIE-2253. Please mention in
> >>> release
> >>> notes.
> >>>
> >>> On Tue, May 26, 2015 at 12:37 PM, Shwetha Shivalingamurthy <
> >>> sshivalingamur...@hortonworks.com> wrote:
> >>>
> >>> > Just one more day for the vote to close. Need atleast 3 votes from
> >>>PMCs.
> >>> > Please review and vote
> >>> >
> >>> > Thanks,
> >>> > Shwetha
> >>> >
> >>> > On 22/05/15 11:30 pm, "Shwetha Shivalingamurthy"
> >>> >  wrote:
> >>> >
> >>> > >Hi All,
> >>> > >
> >>> > >I have created a build for Oozie 4.2.0 (candidate 0).
> >>> > >
> >>> > >Please download, test, try it out and vote:
> >>> > >http://people.apache.org/~shwethags/oozie-4.2.0-rc0/
> >>> > >
> >>> > >The release, md5 signature, gpg signature, rat report, and gpg keys
> >>>can
> >>> > >all be found at the above address.
> >>> > >
> >>> > >Vote closes on Wednesday, May 27th, 2015 12:00 PM GMT
> >>> > >
> >>> > >Thank you all for your contribution
> >>> > >
> >>> > >Regards,
> >>> > >Shwetha
> >>> >
> >>> >
> >>>
> >>>
> >>> --
> >>> Regards
> >>> Pavan Kumar Kolamuri
> >>>
> >
>
>
>
> --
> Harsh J
>

Re: [VOTE] Release Oozie 4.2.0 (candidate 0)

[Rohini Palaniswamy]

Bowen,
Question is whether it is a blocker to the release and requires re-spin
? Bundle logs are only used for debugging in case of a problem and there is
no functionality impact for the user. It is still available on the server
machine. And it only affects the case of automatic state transition and
logs are not affected for user initiated kill/suspend/pause/resume. In my
opinion it can go into the next patch release.

Regards,
Rohini

On Wed, Jun 3, 2015 at 9:20 AM, bowen zhang  wrote:

> log API for bundle relies on lastModifiedTime to set the ending time of
> the searching for oozie.log. With the current bug, lastModifiedTime is set
> prematurely and as a result, log API for bundle job will miss later parts
> of the logs. If you run a simple bundle job right now, and after the job is
> done, you make a api log call, you won't see something like "update bundle
> job status from running to succeeded" which is supposedly be included in
> the log api results. Purshatom created a bug already and I gave a plus one.
> So, he should be able to commit it today. Purshatom, can you comment on the
> status of the bug?Bowen
>
>   From: Shwetha Shivalingamurthy 
>  To: bowen zhang ; Purshotam Shah <
> purus...@yahoo-inc.com>
> Cc: "dev@oozie.apache.org" 
>  Sent: Wednesday, June 3, 2015 3:50 AM
>  Subject: Re: [VOTE] Release Oozie 4.2.0 (candidate 0)
>
> Bowen,
>
> Did you mean BundleEngine.streamJobLog()? I didn’t understand how bundle
> last modified time is related to log API. Doesn’t it just affect updating
> bundle status?
>
> Even if its related to log API, is this blocker for 4.2 release Bowen? The
> issue can be addressed on trunk. Lets take a call on either going ahead
> with the release or cancelling the already voted release.
>
> Thanks,
> Shwetha
>
>
>
>
> On 02/06/15 6:13 am, "bowen zhang" 
> wrote:
>
> >Purshotam,The issue with this one is that log api for bundle jobs no
> >longer runs correctly since it relies on the lastModifiedTime of the job
> >
> >  From: Purshotam Shah 
> > To: "dev@oozie.apache.org" ; bowen zhang
> >
> > Sent: Monday, June 1, 2015 5:24 PM
> > Subject: Re: [VOTE] Release Oozie 4.2.0 (candidate 0)
> >
> >>In BundleStatusTransitService.java method updateJobStatus, shouldn't we
> >>set the lastModifiedTime >for the job as well?Purshatom, can you comment
> >>on my second point since you created the file?
> >Yes, it should. Created https://issues.apache.org/jira/browse/OOZIE-2257
> >to track this.
> >I don't think it's a blocker for 4.2. We are running Oozie with modified
> >StatusTransitService in our prod for more than 6 months. No issue.
> >
> >
> >
> >  From: bowen zhang 
> > To: "dev@oozie.apache.org" 
> > Sent: Sunday, May 31, 2015 1:02 PM
> > Subject: Re: [VOTE] Release Oozie 4.2.0 (candidate 0)
> >
> >I have two concerns.1. In our install doc, it looks like we are still
> >telling user to do "hadoop fs -put share share" which is not correct
> >anymore or at elast not the most updated method2. In
> >BundleStatusTransitService.java method updateJobStatus, shouldn't we set
> >the lastModifiedTime for the job as well?Purshatom, can you comment on my
> >second point since you created the file?Thanks,Bowen
> >
> >  From: Rohini Palaniswamy 
> >
> >
> > To: "dev@oozie.apache.org" 
> > Sent: Thursday, May 28, 2015 12:16 PM
> > Subject: Re: [VOTE] Release Oozie 4.2.0 (candidate 0)
> >
> >+1. Checked signatures. Compiled and ran unit tests.
> >
> >Regards,
> >Rohini
> >
> >On Wed, May 27, 2015 at 2:16 AM, Harsh J  wrote:
> >
> >> +1. I checked the MD5 and the RAT report, and built it
> >> successfully with the -Phadoop-2 flag.
> >>
> >> On Wed, May 27, 2015 at 10:09 AM, Shwetha Shivalingamurthy
> >>  wrote:
> >> > Thanks Robert for verifying.
> >> >
> >> > I didn¹t realise it was a long weekend in the US. Extending the vote
> >>by 2
> >> > days. Vote now closes on Friday, May 29th, 2015 12:00 PM GMT. Please
> >> > verify and vote.
> >> >
> >> > +1 from me.
> >> >
> >> > Thanks,
> >> > Shwetha
> >> >
> >> > On 27/05/15 3:15 am, "Robert Kanter"  wrote:
> >> >
> >> >>+1
> >> >>
> >> >>
> >> >>+ Verified md5
> >> >>+ RAT report looks good
> >> >>+ Ran some of the example

Re: [VOTE] Release Oozie 4.2.0 (candidate 0)

[Rohini Palaniswamy]

The previous section mentions

sharelib create -fs  [-locallib ]
sharelib upgrade -fs  [-locallib ]

and also gives all the details on what those commands do. So
http://oozie.apache.org/docs/4.1.0/DG_QuickStart.html#Oozie_Share_Lib_Installation
 section became redundant when sharelib command was first added to
oozie-setup.sh by you in
https://issues.apache.org/jira/browse/OOZIE-1054. So yes, we need to
get that removed from documentation.  It has been there for a couple
of releases from Oozie 3.3.2 and as doing fs -put share does also
stilll work, I do not have problems either way - it being fixed in
this release or going into next one.  Please feel free to provide a
patch and fix it and if you consider it a blocker work with Shwetha on
getting the patch into this release and re-spinning the release.

Regards,

Rohini

On Wed, Jun 3, 2015 at 9:56 AM, bowen zhang  wrote:

> then,this is not a blocker. What about our docs still saying "hadoop fs
> -put share share"?
>
>   From: Rohini Palaniswamy 
>  To: "dev@oozie.apache.org" ; bowen zhang <
> bowenzhang...@yahoo.com>
> Cc: Shwetha Shivalingamurthy ;
> Purshotam Shah 
>  Sent: Wednesday, June 3, 2015 9:46 AM
>  Subject: Re: [VOTE] Release Oozie 4.2.0 (candidate 0)
>
> Bowen,
> Question is whether it is a blocker to the release and requires re-spin
> ? Bundle logs are only used for debugging in case of a problem and there is
> no functionality impact for the user. It is still available on the server
> machine. And it only affects the case of automatic state transition and
> logs are not affected for user initiated kill/suspend/pause/resume. In my
> opinion it can go into the next patch release.
>
> Regards,
> Rohini
>
>
>
> On Wed, Jun 3, 2015 at 9:20 AM, bowen zhang
>  > wrote:
>
> > log API for bundle relies on lastModifiedTime to set the ending time of
> > the searching for oozie.log. With the current bug, lastModifiedTime is
> set
> > prematurely and as a result, log API for bundle job will miss later parts
> > of the logs. If you run a simple bundle job right now, and after the job
> is
> > done, you make a api log call, you won't see something like "update
> bundle
> > job status from running to succeeded" which is supposedly be included in
> > the log api results. Purshatom created a bug already and I gave a plus
> one.
> > So, he should be able to commit it today. Purshatom, can you comment on
> the
> > status of the bug?Bowen
> >
> >  From: Shwetha Shivalingamurthy 
> >  To: bowen zhang ; Purshotam Shah <
> > purus...@yahoo-inc.com>
> > Cc: "dev@oozie.apache.org" 
> >  Sent: Wednesday, June 3, 2015 3:50 AM
> >  Subject: Re: [VOTE] Release Oozie 4.2.0 (candidate 0)
> >
> > Bowen,
> >
> > Did you mean BundleEngine.streamJobLog()? I didn’t understand how bundle
> > last modified time is related to log API. Doesn’t it just affect updating
> > bundle status?
> >
> > Even if its related to log API, is this blocker for 4.2 release Bowen?
> The
> > issue can be addressed on trunk. Lets take a call on either going ahead
> > with the release or cancelling the already voted release.
> >
> > Thanks,
> > Shwetha
> >
> >
> >
> >
> > On 02/06/15 6:13 am, "bowen zhang" 
> > wrote:
> >
> > >Purshotam,The issue with this one is that log api for bundle jobs no
> > >longer runs correctly since it relies on the lastModifiedTime of the job
> > >
> > >  From: Purshotam Shah 
> > > To: "dev@oozie.apache.org" ; bowen zhang
> > >
> > > Sent: Monday, June 1, 2015 5:24 PM
> > > Subject: Re: [VOTE] Release Oozie 4.2.0 (candidate 0)
> > >
> > >>In BundleStatusTransitService.java method updateJobStatus, shouldn't we
> > >>set the lastModifiedTime >for the job as well?Purshatom, can you
> comment
> > >>on my second point since you created the file?
> > >Yes, it should. Created
> https://issues.apache.org/jira/browse/OOZIE-2257
> > >to track this.
> > >I don't think it's a blocker for 4.2. We are running Oozie with modified
> > >StatusTransitService in our prod for more than 6 months. No issue.
> > >
> > >
> > >
> > >  From: bowen zhang 
> > > To: "dev@oozie.apache.org" 
> > > Sent: Sunday, May 31, 2015 1:02 PM
> > > Subject: Re: [VOTE] Release Oozie 4.2.0 (candidate 0)
> > >
> > >I have two concerns.1. In our install doc, it looks like we are still
> > >telling user to do "hadoop fs -put share share

Re: Review Request 33400: OOZIE-2215 Support glob in FS EL function

[Rohini Palaniswamy]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/33400/#review87085
---



core/src/main/java/org/apache/oozie/action/hadoop/FsELFunctions.java
<https://reviews.apache.org/r/33400/#comment139363>

exists variable not required

return (pathArr != null && pathArr.length > 0);



docs/src/site/twiki/WorkflowFunctionalSpec.twiki
<https://reviews.apache.org/r/33400/#comment139362>

...when there is at least one ...


- Rohini Palaniswamy


On June 1, 2015, 5:49 p.m., Ryota Egashira wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/33400/
> ---
> 
> (Updated June 1, 2015, 5:49 p.m.)
> 
> 
> Review request for oozie.
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> https://issues.apache.org/jira/browse/OOZIE-2215
> 
> 
> Diffs
> -
> 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsELFunctions.java 
> f8af61c 
>   core/src/main/resources/oozie-default.xml 8960073 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestFsELFunctions.java 
> c81fc61 
>   docs/src/site/twiki/WorkflowFunctionalSpec.twiki 02dc65b 
> 
> Diff: https://reviews.apache.org/r/33400/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Ryota Egashira
> 
>

Review of pending patches

[Rohini Palaniswamy]

Committers,
I see that patches submitted by users have been piling up. Can all of
you pitch in and help review the below ones and get it committed? If the
patches are not relevant, clear the Patch available so that it gets removed
from this report.

Regards,
Rohini

-- Forwarded message --
From: 
Date: Thu, Jun 11, 2015 at 11:00 PM
Subject: [jira] Subscription: Oozie Patch Available
To: dev@oozie.apache.org


Issue Subscription
Filter: Oozie Patch Available (57 issues)

Subscriber: ooziedaily

Key Summary
OOZIE-2268  Update ActiveMQ version for security and other fixes
https://issues.apache.org/jira/browse/OOZIE-2268
OOZIE-2266  [core] Fix 'total' actions returned in coordinator job info
https://issues.apache.org/jira/browse/OOZIE-2266
OOZIE-2264  Fix coord:offset(n,"DAY") to resolve correct data set
https://issues.apache.org/jira/browse/OOZIE-2264
OOZIE-2262  Fix log streaming from other server with start/end filter
https://issues.apache.org/jira/browse/OOZIE-2262
OOZIE-2258  Introducing a new counter in the instrumentation log to
distinguish between the reasons for launcher failure
https://issues.apache.org/jira/browse/OOZIE-2258
OOZIE-2257  BundleStatusTransitXCommand doesn't update last modified time.
https://issues.apache.org/jira/browse/OOZIE-2257
OOZIE-2251  Expose instrumental matrices in Realtime Graphing tool
https://issues.apache.org/jira/browse/OOZIE-2251
OOZIE-2250  Show log for WAITING and TIMEDOUT coord actions
https://issues.apache.org/jira/browse/OOZIE-2250
OOZIE-2245  Service to periodically check database schema
https://issues.apache.org/jira/browse/OOZIE-2245
OOZIE-2244  Oozie should mask passwords in the logs when logging command
arguments
https://issues.apache.org/jira/browse/OOZIE-2244
OOZIE-2243  Kill Command does not kill the child job for java action
https://issues.apache.org/jira/browse/OOZIE-2243
OOZIE-2215  Support glob in FS EL function
https://issues.apache.org/jira/browse/OOZIE-2215
OOZIE-2203  Fix the login example
https://issues.apache.org/jira/browse/OOZIE-2203
OOZIE-2196  Create Local Client for Bundle
https://issues.apache.org/jira/browse/OOZIE-2196
OOZIE-2187  Add a way to specify a default JT/RM and NN
https://issues.apache.org/jira/browse/OOZIE-2187
OOZIE-2178  fix javadoc to compile on JDK8
https://issues.apache.org/jira/browse/OOZIE-2178
OOZIE-2168  Oozie flow and action names have 50 char limit
https://issues.apache.org/jira/browse/OOZIE-2168
OOZIE-2159  'oozie validate' command should be moved server-side
https://issues.apache.org/jira/browse/OOZIE-2159
OOZIE-2134  Remove references to Services.get().getConf() in code
https://issues.apache.org/jira/browse/OOZIE-2134
OOZIE-2106  Make tomcat download url configurable in the pom file
https://issues.apache.org/jira/browse/OOZIE-2106
OOZIE-2105  Make version of submodules configurable with parent version
https://issues.apache.org/jira/browse/OOZIE-2105
OOZIE-2099  Add test-patch support for patches generated without --no-prefix
https://issues.apache.org/jira/browse/OOZIE-2099
OOZIE-2081  WorkflowJob notification to include coordinator action id
https://issues.apache.org/jira/browse/OOZIE-2081
OOZIE-2060  Incorrect documentation of Java action config XML filename
https://issues.apache.org/jira/browse/OOZIE-2060
OOZIE-2044  ssh action succeed with a not exists command which should be
fail.
https://issues.apache.org/jira/browse/OOZIE-2044
OOZIE-2020  Rerun all Failed/killed/timedout coordinator actions rather
than specifying action numbers
https://issues.apache.org/jira/browse/OOZIE-2020
OOZIE-1980  Sql error should not fail coord job
https://issues.apache.org/jira/browse/OOZIE-1980
OOZIE-1977  Display patch analysis issues
https://issues.apache.org/jira/browse/OOZIE-1977
OOZIE-1936  Queuedump command should display queue information for all
server.
https://issues.apache.org/jira/browse/OOZIE-1936
OOZIE-1931  Admin command to print all locks held by server(s)
https://issues.apache.org/jira/browse/OOZIE-1931
OOZIE-1927  Use StoreStatusFilter for WorkflowsJobGetJPAExecutor
https://issues.apache.org/jira/browse/OOZIE-1927
OOZIE-1922  MemoryLocksService fails if lock is acquired multiple times in
same thread and released
https://issues.apache.org/jira/browse/OOZIE-1922
OOZIE-1918  ActionXCommand refactoring for code reuse
https://issues.apache.org/jira/browse/OOZIE-1918
OOZIE-1884  ZKJobsConcurrencyService.checkJobIdForServer fails if jobs
length > 7
https://issues.apache.org/jira/browse/OOZIE-1884
OOZIE-1860  Oozie job mapper launch fails due to null value returned from
action file
ht

Re: Review Request 35340: OOZIE-2187: Add a way to specify a default JT/RM and NN

[Rohini Palaniswamy]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/35340/#review87765
---



core/src/main/java/org/apache/oozie/workflow/lite/LiteWorkflowAppParser.java
<https://reviews.apache.org/r/35340/#comment140183>

This method does not seem to be called from anywhere



core/src/main/java/org/apache/oozie/workflow/lite/LiteWorkflowAppParser.java
<https://reviews.apache.org/r/35340/#comment140184>

Wouldn't it be more performant to parse each of the jobxml once and store 
them in Map so that we don't parse them again for 
each action?


- Rohini Palaniswamy


On June 11, 2015, 2:10 a.m., Robert Kanter wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/35340/
> ---
> 
> (Updated June 11, 2015, 2:10 a.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-2187
> https://issues.apache.org/jira/browse/OOZIE-2187
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> See this comment:
> https://issues.apache.org/jira/browse/OOZIE-2187?focusedCommentId=14522592&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14522592
> 
> 
> Diffs
> -
> 
>   core/src/main/java/org/apache/oozie/action/ActionExecutor.java ea652f3 
>   core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 
> 7a7d4a7 
>   
> core/src/main/java/org/apache/oozie/workflow/lite/LiteWorkflowAppParser.java 
> c857011 
>   core/src/main/resources/oozie-default.xml 8960073 
>   
> core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java
>  0eb1ee0 
>   core/src/test/resources/wf-schema-no-jobtracker-global.xml PRE-CREATION 
>   core/src/test/resources/wf-schema-no-jobtracker.xml PRE-CREATION 
>   core/src/test/resources/wf-schema-no-namenode-global.xml PRE-CREATION 
>   core/src/test/resources/wf-schema-no-namenode.xml PRE-CREATION 
>   core/src/test/resources/wf-schema-valid-global-ext.xml 86d274d 
> 
> Diff: https://reviews.apache.org/r/35340/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Robert Kanter
> 
>

Re: Jenkins is currently unable to login to JIRA

[Rohini Palaniswamy]

+Giri who might know better.

On Wed, Jun 17, 2015 at 5:35 PM, Robert Kanter  wrote:

> Recent builds have been failing, and not posting to JIRA because of this:
>
> Adding comment to JIRA
> Unable to log in to server:
> https://issues.apache.org/jira/rpc/soap/jirasoapservice-v2 with user:
> hadoopqa.
>  Cause: ; nested exception is:
> javax.net.ssl.SSLException: java.lang.RuntimeException: Could not generate
> DH keypair
>
> test-patch exit code: 1
>
> (Taken from the end of
> https://builds.apache.org/job/oozie-trunk-precommit-build/2419/console)
>
> Any ideas?
>
>
> - Robert
>

Re: Review Request 35340: OOZIE-2187: Add a way to specify a default JT/RM and NN

[Rohini Palaniswamy]

> On June 12, 2015, 10:16 p.m., Rohini Palaniswamy wrote:
> > core/src/main/java/org/apache/oozie/workflow/lite/LiteWorkflowAppParser.java,
> >  line 638
> > <https://reviews.apache.org/r/35340/diff/1/?file=982401#file982401line638>
> >
> > Wouldn't it be more performant to parse each of the jobxml once and 
> > store them in Map so that we don't parse them 
> > again for each action?
> 
> Robert Kanter wrote:
> The code here is simply adding a  in the action, without 
> actually parsing the referenced file; that's done later in the 
> ActionExecutor.  
> 
> If we want to parse  here instead, I think that should be saved 
> for another JIRA; we'd have to think about if we actually want to do this 
> because it will have other reprecussions (e.g. it will take up more space in 
> the database).

Sounds good. +1 from my side. Jaydeep had some minor comments on using static 
string for job-xml and formatting. Will +1 here and in the jira after that is 
addressed.


- Rohini


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/35340/#review87765
---


On June 23, 2015, 8:58 p.m., Robert Kanter wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/35340/
> ---
> 
> (Updated June 23, 2015, 8:58 p.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-2187
> https://issues.apache.org/jira/browse/OOZIE-2187
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> See this comment:
> https://issues.apache.org/jira/browse/OOZIE-2187?focusedCommentId=14522592&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14522592
> 
> 
> Diffs
> -
> 
>   core/src/main/java/org/apache/oozie/action/ActionExecutor.java 07c6d26 
>   core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 
> 492ceaf 
>   
> core/src/main/java/org/apache/oozie/workflow/lite/LiteWorkflowAppParser.java 
> c857011 
>   core/src/main/resources/oozie-default.xml 4dc127e 
>   
> core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java
>  0eb1ee0 
>   core/src/test/resources/wf-schema-no-jobtracker-global.xml PRE-CREATION 
>   core/src/test/resources/wf-schema-no-jobtracker.xml PRE-CREATION 
>   core/src/test/resources/wf-schema-no-namenode-global.xml PRE-CREATION 
>   core/src/test/resources/wf-schema-no-namenode.xml PRE-CREATION 
>   core/src/test/resources/wf-schema-valid-global-ext.xml 86d274d 
> 
> Diff: https://reviews.apache.org/r/35340/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Robert Kanter
> 
>

Re: Review Request 35340: OOZIE-2187: Add a way to specify a default JT/RM and NN

[Rohini Palaniswamy]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/35340/#review91299
---

Ship it!


Ship It!

- Rohini Palaniswamy


On July 9, 2015, 10:56 p.m., Robert Kanter wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/35340/
> ---
> 
> (Updated July 9, 2015, 10:56 p.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-2187
> https://issues.apache.org/jira/browse/OOZIE-2187
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> See this comment:
> https://issues.apache.org/jira/browse/OOZIE-2187?focusedCommentId=14522592&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14522592
> 
> 
> Diffs
> -
> 
>   core/src/main/java/org/apache/oozie/action/ActionExecutor.java 07c6d26 
>   core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 
> 492ceaf 
>   
> core/src/main/java/org/apache/oozie/workflow/lite/LiteWorkflowAppParser.java 
> c857011 
>   core/src/main/resources/oozie-default.xml 4dc127e 
>   
> core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java
>  0eb1ee0 
>   core/src/test/resources/wf-schema-no-jobtracker-global.xml PRE-CREATION 
>   core/src/test/resources/wf-schema-no-jobtracker.xml PRE-CREATION 
>   core/src/test/resources/wf-schema-no-namenode-global.xml PRE-CREATION 
>   core/src/test/resources/wf-schema-no-namenode.xml PRE-CREATION 
>   core/src/test/resources/wf-schema-valid-global-ext.xml 86d274d 
> 
> Diff: https://reviews.apache.org/r/35340/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Robert Kanter
> 
>

Re: Jenkins is currently unable to login to JIRA

[Rohini Palaniswamy]

Giri,
   Thanks. I was looking at the jenkins config yesterday but had missed the
JDK section. Was any other change required other than setting JDK dropdown
to JDK 1.7 (latest) ?

Thanks,
Rohini

On Thu, Jul 9, 2015 at 4:46 PM, Giridharan Kesavan  wrote:

> Updated the job config to point to jdk1.7 latest.
>
> From: Ryota Egashira
> Date: Thursday, July 9, 2015 at 3:24 PM
> To: Giridharan Kesavan, "dev@oozie.apache.org<mailto:dev@oozie.apache.org
> >"
> Subject: Re: Jenkins is currently unable to login to JIRA
>
> Giri
>
> https://issues.apache.org/jira/browse/INFRA-9948
>
> "you'll need to configure the build to use Java 7, and possibly configure
> the build to target Java 6.  Java 6 is no longer compatible with the
> updated openssl and httpd deployment on the SSL proxy serving
> issues.apache.org"
>
>
> how to make JAVA_HOME point to Java 7?
>
>
>
>
> On Tuesday, July 7, 2015 11:48 PM, Giridharan Kesavan <
> gkesa...@hortonworks.com<mailto:gkesa...@hortonworks.com>> wrote:
>
>
> Somehow I missed this email. Will look into it in the morning
>
> Sent from Outlook<http://taps.io/outlookmobile>
>
>
>
>
> On Tue, Jul 7, 2015 at 3:55 PM -0700, "Ryota Egashira" <
> egash...@yahoo-inc.com<mailto:egash...@yahoo-inc.com>> wrote:
>
> this is still an issue. opened INFRA ticket.
> https://issues.apache.org/jira/browse/INFRA-9948
>
> Ryota
>
>
>
> On Monday, June 22, 2015 12:52 PM, Rohini Palaniswamy <
> rohini.adi...@gmail.com<mailto:rohini.adi...@gmail.com>> wrote:
>
>
> +Giri who might know better.
>
> On Wed, Jun 17, 2015 at 5:35 PM, Robert Kanter  <mailto:rkan...@cloudera.com>> wrote:
>
> > Recent builds have been failing, and not posting to JIRA because of this:
> >
> > Adding comment to JIRA
> > Unable to log in to server:
> > https://issues.apache.org/jira/rpc/soap/jirasoapservice-v2 with user:
> > hadoopqa.
> >  Cause: ; nested exception is:
> > javax.net.ssl.SSLException: java.lang.RuntimeException: Could not
> generate
> > DH keypair
> >
> > test-patch exit code: 1
> >
> > (Taken from the end of
> > https://builds.apache.org/job/oozie-trunk-precommit-build/2419/console)
> >
> > Any ideas?
> >
> >
> > - Robert
> >
>
>
>
>
>

Re: Review Request 35379: OOZIE-2030 Configuration properties from global section is not getting set in Hadoop job conf when using sub-workflow action in Oozie workflow.xml

[Rohini Palaniswamy]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/35379/#review91987
---


OOZIE-2187 changes this piece of code a lot. You will have to rebase and make 
changes to work with that.


core/src/main/java/org/apache/oozie/workflow/lite/LiteWorkflowAppParser.java 
(line 445)
<https://reviews.apache.org/r/35379/#comment145763>

Can you also add condition to check for presence of propagate-configuration?


- Rohini Palaniswamy


On July 14, 2015, 1:19 p.m., Jaydeep Vishwakarma wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/35379/
> ---
> 
> (Updated July 14, 2015, 1:19 p.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-2030
> https://issues.apache.org/jira/browse/OOZIE-2030
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> Forwarding global conf from workflow to action when action is subworkflow, 
> subworkflow storing it to property and forwarding as property. 
>  LiteworkflowAppParser handling all condition and ensuring child level pass 
> of global conf.
> 
> 
> Diffs
> -
> 
>   
> core/src/main/java/org/apache/oozie/workflow/lite/LiteWorkflowAppParser.java 
> c857011 
>   
> core/src/test/java/org/apache/oozie/action/oozie/TestSubWorkflowActionExecutor.java
>  9ab897a 
> 
> Diff: https://reviews.apache.org/r/35379/diff/
> 
> 
> Testing
> ---
> 
> yes
> 
> 
> Thanks,
> 
> Jaydeep Vishwakarma
> 
>

Re: Review Request 37452: OOZIE-2277: Honor oozie.action.sharelib.for.spark in Spark jobs

[Rohini Palaniswamy]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37452/#review95987
---



core/src/main/java/org/apache/oozie/service/SparkConfigurationService.java 
(line 85)
<https://reviews.apache.org/r/37452/#comment151180>

Should we be doing this? From 
http://spark.apache.org/docs/latest/running-on-yarn.html ,it refers to hdfs 
location or local installation in task node. Since it applies for other 
clients, should we retain that in Oozie as well or we are saying that Oozie is 
only going to use spark libraries via sharelib?  Or atleast it should be 
configurable to retain this setting to support users have local installation in 
task nodes.



sharelib/spark/pom.xml (line 121)
<https://reviews.apache.org/r/37452/#comment151181>

Nitpick. Can you put spark-core in the beginning followed by other spark 
feature dependencies as that is the main one.



sharelib/spark/src/main/java/org.apache.oozie.action.hadoop/SparkMain.java 
(lines 60 - 61)
<https://reviews.apache.org/r/37452/#comment152402>

Robert, 
   In our chat, you mentioned about ability to specify this alternatively 
as -master yarn -mode client and -master yarn -mode cluster. Will have to 
handle that as well.



sharelib/spark/src/main/java/org.apache.oozie.action.hadoop/SparkMain.java 
(line 78)
<https://reviews.apache.org/r/37452/#comment152398>

If local mode will be ever used in Oozie, then all the new code can go into 
a if (yarnClusterMode || yarnClientMode) block to be done only for non-local 
mode.



sharelib/spark/src/main/java/org.apache.oozie.action.hadoop/SparkMain.java 
(line 103)
<https://reviews.apache.org/r/37452/#comment152407>

DELIM is space which means we expect user to specify exactly one space 
between the arguments. We should do \s+ or we can try something like 
http://stackoverflow.com/questions/6049470/can-apache-commons-cli-options-parser-ignore-unknown-command-line-options/8613949#8613949
 to be more cleaner with parsing the arguments.



sharelib/spark/src/main/java/org.apache.oozie.action.hadoop/SparkMain.java 
(line 113)
<https://reviews.apache.org/r/37452/#comment152417>

Can you place local files also in spark.yarn.dist.files and spark takes 
care of shipping them like --jars? Asking because you are adding files from 
java.classpath to sparkJars. Atleast in hadoop mapreduce.cache.files have to be 
hdfs paths.



sharelib/spark/src/main/java/org.apache.oozie.action.hadoop/SparkMain.java 
(line 123)
<https://reviews.apache.org/r/37452/#comment152408>

Can you add just add a comment here saying this is redundant for 
yarnClientMode as driver is the launcher jvm and it is already launched.


Robert did enlighten me that

"in local mode, everything runs in the launcher job. in yarn-client mode, the 
driver runs in the launcher and the executor in Yarn. in yarn-cluster mode, the 
driver and executor run in Yarn"

Can we added that to code comments and also in documentation as it will be 
confusing for users as well.

- Rohini Palaniswamy


On Aug. 13, 2015, 11:42 p.m., Robert Kanter wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/37452/
> ---
> 
> (Updated Aug. 13, 2015, 11:42 p.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-2277
> https://issues.apache.org/jira/browse/OOZIE-2277
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> https://issues.apache.org/jira/browse/OOZIE-2277
> 
> 
> Diffs
> -
> 
>   core/src/main/java/org/apache/oozie/service/SparkConfigurationService.java 
> 1b7cf4a 
>   
> core/src/test/java/org/apache/oozie/service/TestSparkConfigurationService.java
>  b2c499d 
>   sharelib/spark/pom.xml 6f7e74a 
>   sharelib/spark/src/main/java/org.apache.oozie.action.hadoop/SparkMain.java 
> b18a0b9 
>   
> sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestSparkActionExecutor.java
>  f271abc 
> 
> Diff: https://reviews.apache.org/r/37452/diff/
> 
> 
> Testing
> ---
> 
> - Ran unit tests with Hadoop 1 and Hadoop 2
> - Ran in a Hadoop 2 cluster with local, yarn-client, and yarn-cluster modes
> 
> 
> Thanks,
> 
> Robert Kanter
> 
>

Re: Review Request 37906: OOZIE-2345 -Concurrent job submission for forked action

[Rohini Palaniswamy]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37906/#review97092
---



core/src/main/java/org/apache/oozie/command/wf/ForkedActionStartXCommand.java 
(line 82)
<https://reviews.apache.org/r/37906/#comment152849>

Add a comment saying not actualling ending here, but setting a flag. 
SignalXCommand will handle



core/src/main/java/org/apache/oozie/command/wf/SignalXCommand.java (line 98)
<https://reviews.apache.org/r/37906/#comment152832>

oozie.workflow.parallel.fork.action.start



core/src/main/java/org/apache/oozie/command/wf/SignalXCommand.java (line 477)
<https://reviews.apache.org/r/37906/#comment152870>

Starting forked actions parallely



core/src/main/java/org/apache/oozie/command/wf/SignalXCommand.java (line 481)
<https://reviews.apache.org/r/37906/#comment152839>

Expose APIs in CallableQueueService executor and submit using that. 
SchedulerService has very few threads.



core/src/test/java/org/apache/oozie/command/wf/TestSignalXCommand.java (line 75)
<https://reviews.apache.org/r/37906/#comment152831>

services.destroy() before new Services() in all tests


- Rohini Palaniswamy


On Aug. 31, 2015, 6:46 p.m., Purshotam Shah wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/37906/
> ---
> 
> (Updated Aug. 31, 2015, 6:46 p.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-2345
> https://issues.apache.org/jira/browse/OOZIE-2345
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> We have few customers whose SLA is 8 min. They have around 30 actions. There 
> are 25 actions in fork.
> Though forked action jobs runs concurrently. forked action job submission is 
> sequential.
> Whenever NN is slow, job submission takes more time. Even if job submission 
> is delay for 30 sec. Total WF delay will be ~12 min.
> 
> 
> Diffs
> -
> 
>   core/src/main/java/org/apache/oozie/command/wf/ActionStartXCommand.java 
> e06649cf9bbc7c0997e1d4f94fa7e695533f8343 
>   core/src/main/java/org/apache/oozie/command/wf/ActionXCommand.java 
> 2616d322ea0dd85942fcb33c73d7d33af049b789 
>   
> core/src/main/java/org/apache/oozie/command/wf/ForkedActionStartXCommand.java 
> e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 
>   core/src/main/java/org/apache/oozie/command/wf/ResumeXCommand.java 
> edbd7679a82abbe9826d4530e489d43baf23564e 
>   core/src/main/java/org/apache/oozie/command/wf/SignalXCommand.java 
> d1fcd1ac933aaeaea22cdd3fcbe55be386281fbb 
>   core/src/main/java/org/apache/oozie/service/RecoveryService.java 
> 4b4a3f2e72c28c2d3afee74c10059d0d60cdf666 
>   core/src/main/resources/oozie-default.xml 
> 32a1df044b9910ba9cfade88005a598e0ab3a7b3 
>   
> core/src/test/java/org/apache/oozie/command/wf/TestForkedActionStartXCommand.java
>  e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 
>   core/src/test/java/org/apache/oozie/command/wf/TestReRunXCommand.java 
> 02f61668e6d6d03674f5d516853057fb94fed518 
>   core/src/test/java/org/apache/oozie/command/wf/TestSignalXCommand.java 
> 4268b301d3efff05e870b3d833ba5fd5b9cc0d5a 
> 
> Diff: https://reviews.apache.org/r/37906/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Purshotam Shah
> 
>

Re: Review Request 37906: OOZIE-2345 -Concurrent job submission for forked action

[Rohini Palaniswamy]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37906/#review97126
---


Since SignalXCommand is holding the workflow lock, you don't need to acquire 
action level locks for Resume, RecoveryService, etc.

- Rohini Palaniswamy


On Aug. 31, 2015, 6:46 p.m., Purshotam Shah wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/37906/
> ---
> 
> (Updated Aug. 31, 2015, 6:46 p.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-2345
> https://issues.apache.org/jira/browse/OOZIE-2345
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> We have few customers whose SLA is 8 min. They have around 30 actions. There 
> are 25 actions in fork.
> Though forked action jobs runs concurrently. forked action job submission is 
> sequential.
> Whenever NN is slow, job submission takes more time. Even if job submission 
> is delay for 30 sec. Total WF delay will be ~12 min.
> 
> 
> Diffs
> -
> 
>   core/src/main/java/org/apache/oozie/command/wf/ActionStartXCommand.java 
> e06649cf9bbc7c0997e1d4f94fa7e695533f8343 
>   core/src/main/java/org/apache/oozie/command/wf/ActionXCommand.java 
> 2616d322ea0dd85942fcb33c73d7d33af049b789 
>   
> core/src/main/java/org/apache/oozie/command/wf/ForkedActionStartXCommand.java 
> e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 
>   core/src/main/java/org/apache/oozie/command/wf/ResumeXCommand.java 
> edbd7679a82abbe9826d4530e489d43baf23564e 
>   core/src/main/java/org/apache/oozie/command/wf/SignalXCommand.java 
> d1fcd1ac933aaeaea22cdd3fcbe55be386281fbb 
>   core/src/main/java/org/apache/oozie/service/RecoveryService.java 
> 4b4a3f2e72c28c2d3afee74c10059d0d60cdf666 
>   core/src/main/resources/oozie-default.xml 
> 32a1df044b9910ba9cfade88005a598e0ab3a7b3 
>   
> core/src/test/java/org/apache/oozie/command/wf/TestForkedActionStartXCommand.java
>  e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 
>   core/src/test/java/org/apache/oozie/command/wf/TestReRunXCommand.java 
> 02f61668e6d6d03674f5d516853057fb94fed518 
>   core/src/test/java/org/apache/oozie/command/wf/TestSignalXCommand.java 
> 4268b301d3efff05e870b3d833ba5fd5b9cc0d5a 
> 
> Diff: https://reviews.apache.org/r/37906/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Purshotam Shah
> 
>

Re: Review Request 37452: OOZIE-2277: Honor oozie.action.sharelib.for.spark in Spark jobs

[Rohini Palaniswamy]

> On Aug. 27, 2015, 10:03 p.m., Rohini Palaniswamy wrote:
> > sharelib/spark/src/main/java/org.apache.oozie.action.hadoop/SparkMain.java, 
> > line 78
> > <https://reviews.apache.org/r/37452/diff/1/?file=1039664#file1039664line78>
> >
> > If local mode will be ever used in Oozie, then all the new code can go 
> > into a if (yarnClusterMode || yarnClientMode) block to be done only for 
> > non-local mode.
> 
> Robert Kanter wrote:
> local mode still requires setting --jars, so I'd have to duplicate that 
> part of the new code in an else statement which I think might be harder to 
> follow/maintain.  Can we leave this as is?

Sure.


- Rohini


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37452/#review95987
---


On Aug. 28, 2015, 5:03 a.m., Robert Kanter wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/37452/
> ---
> 
> (Updated Aug. 28, 2015, 5:03 a.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-2277
> https://issues.apache.org/jira/browse/OOZIE-2277
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> https://issues.apache.org/jira/browse/OOZIE-2277
> 
> 
> Diffs
> -
> 
>   core/src/main/java/org/apache/oozie/service/SparkConfigurationService.java 
> 1b7cf4a 
>   core/src/main/resources/oozie-default.xml 32a1df0 
>   
> core/src/test/java/org/apache/oozie/service/TestSparkConfigurationService.java
>  b2c499d 
>   docs/src/site/twiki/DG_SparkActionExtension.twiki 32ebe12 
>   sharelib/spark/pom.xml 6f7e74a 
>   sharelib/spark/src/main/java/org.apache.oozie.action.hadoop/SparkMain.java 
> b18a0b9 
>   
> sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestSparkActionExecutor.java
>  f271abc 
> 
> Diff: https://reviews.apache.org/r/37452/diff/
> 
> 
> Testing
> ---
> 
> - Ran unit tests with Hadoop 1 and Hadoop 2
> - Ran in a Hadoop 2 cluster with local, yarn-client, and yarn-cluster modes
> 
> 
> Thanks,
> 
> Robert Kanter
> 
>

Re: Review Request 37452: OOZIE-2277: Honor oozie.action.sharelib.for.spark in Spark jobs

[Rohini Palaniswamy]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37452/#review97166
---



sharelib/spark/pom.xml 
<https://reviews.apache.org/r/37452/#comment152949>

Are we removing spark-core altogether as other dependencies will bring it?


- Rohini Palaniswamy


On Aug. 28, 2015, 5:03 a.m., Robert Kanter wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/37452/
> ---
> 
> (Updated Aug. 28, 2015, 5:03 a.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-2277
> https://issues.apache.org/jira/browse/OOZIE-2277
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> https://issues.apache.org/jira/browse/OOZIE-2277
> 
> 
> Diffs
> -
> 
>   core/src/main/java/org/apache/oozie/service/SparkConfigurationService.java 
> 1b7cf4a 
>   core/src/main/resources/oozie-default.xml 32a1df0 
>   
> core/src/test/java/org/apache/oozie/service/TestSparkConfigurationService.java
>  b2c499d 
>   docs/src/site/twiki/DG_SparkActionExtension.twiki 32ebe12 
>   sharelib/spark/pom.xml 6f7e74a 
>   sharelib/spark/src/main/java/org.apache.oozie.action.hadoop/SparkMain.java 
> b18a0b9 
>   
> sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestSparkActionExecutor.java
>  f271abc 
> 
> Diff: https://reviews.apache.org/r/37452/diff/
> 
> 
> Testing
> ---
> 
> - Ran unit tests with Hadoop 1 and Hadoop 2
> - Ran in a Hadoop 2 cluster with local, yarn-client, and yarn-cluster modes
> 
> 
> Thanks,
> 
> Robert Kanter
> 
>

Re: Review Request 37452: OOZIE-2277: Honor oozie.action.sharelib.for.spark in Spark jobs

[Rohini Palaniswamy]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37452/#review97167
---

Ship it!


Ship It!

- Rohini Palaniswamy


On Aug. 28, 2015, 5:03 a.m., Robert Kanter wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/37452/
> ---
> 
> (Updated Aug. 28, 2015, 5:03 a.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-2277
> https://issues.apache.org/jira/browse/OOZIE-2277
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> https://issues.apache.org/jira/browse/OOZIE-2277
> 
> 
> Diffs
> -
> 
>   core/src/main/java/org/apache/oozie/service/SparkConfigurationService.java 
> 1b7cf4a 
>   core/src/main/resources/oozie-default.xml 32a1df0 
>   
> core/src/test/java/org/apache/oozie/service/TestSparkConfigurationService.java
>  b2c499d 
>   docs/src/site/twiki/DG_SparkActionExtension.twiki 32ebe12 
>   sharelib/spark/pom.xml 6f7e74a 
>   sharelib/spark/src/main/java/org.apache.oozie.action.hadoop/SparkMain.java 
> b18a0b9 
>   
> sharelib/spark/src/test/java/org/apache/oozie/action/hadoop/TestSparkActionExecutor.java
>  f271abc 
> 
> Diff: https://reviews.apache.org/r/37452/diff/
> 
> 
> Testing
> ---
> 
> - Ran unit tests with Hadoop 1 and Hadoop 2
> - Ran in a Hadoop 2 cluster with local, yarn-client, and yarn-cluster modes
> 
> 
> Thanks,
> 
> Robert Kanter
> 
>

Re: Review Request 37906: OOZIE-2345 -Concurrent job submission for forked action

[Rohini Palaniswamy]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37906/#review97303
---



core/src/main/java/org/apache/oozie/command/wf/ActionStartXCommand.java (line 
355)
<https://reviews.apache.org/r/37906/#comment153145>

Can be reverted. This is not required as ActionEndXCommand is called from 
ActionStartXCommand only for control nodes like start, decision, fork and join. 
It can be enhanced to do for fs actions. For all others, ActionEndXCommand will 
be called by ActionCheckXCommand.


- Rohini Palaniswamy


On Sept. 1, 2015, 4:46 p.m., Purshotam Shah wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/37906/
> ---
> 
> (Updated Sept. 1, 2015, 4:46 p.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-2345
> https://issues.apache.org/jira/browse/OOZIE-2345
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> We have few customers whose SLA is 8 min. They have around 30 actions. There 
> are 25 actions in fork.
> Though forked action jobs runs concurrently. forked action job submission is 
> sequential.
> Whenever NN is slow, job submission takes more time. Even if job submission 
> is delay for 30 sec. Total WF delay will be ~12 min.
> 
> 
> Diffs
> -
> 
>   core/src/main/java/org/apache/oozie/command/wf/ActionStartXCommand.java 
> e06649cf9bbc7c0997e1d4f94fa7e695533f8343 
>   core/src/main/java/org/apache/oozie/command/wf/ActionXCommand.java 
> 2616d322ea0dd85942fcb33c73d7d33af049b789 
>   
> core/src/main/java/org/apache/oozie/command/wf/ForkedActionStartXCommand.java 
> e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 
>   core/src/main/java/org/apache/oozie/command/wf/SignalXCommand.java 
> d1fcd1ac933aaeaea22cdd3fcbe55be386281fbb 
>   core/src/main/java/org/apache/oozie/service/CallableQueueService.java 
> 830a58ebe152baed0a922e78855be51d3bf1b21c 
>   core/src/main/resources/oozie-default.xml 
> 32a1df044b9910ba9cfade88005a598e0ab3a7b3 
>   
> core/src/test/java/org/apache/oozie/command/wf/TestForkedActionStartXCommand.java
>  e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 
>   core/src/test/java/org/apache/oozie/command/wf/TestReRunXCommand.java 
> 02f61668e6d6d03674f5d516853057fb94fed518 
>   core/src/test/java/org/apache/oozie/command/wf/TestSignalXCommand.java 
> 4268b301d3efff05e870b3d833ba5fd5b9cc0d5a 
> 
> Diff: https://reviews.apache.org/r/37906/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Purshotam Shah
> 
>

Re: Review Request 37906: OOZIE-2345 -Concurrent job submission for forked action

[Rohini Palaniswamy]

> On Sept. 1, 2015, 5 p.m., Rohini Palaniswamy wrote:
> > core/src/main/java/org/apache/oozie/command/wf/ActionStartXCommand.java, 
> > line 357
> > <https://reviews.apache.org/r/37906/diff/1-3/?file=1058385#file1058385line357>
> >
> > Can be reverted. This is not required as ActionEndXCommand is called 
> > from ActionStartXCommand only for control nodes like start, decision, fork 
> > and join. It can be enhanced to do for fs actions. For all others, 
> > ActionEndXCommand will be called by ActionCheckXCommand.

Actually leave it as it is. If we enchance fs actions to end immediately that 
would come in handy when fs actions are in the fork or even now if fork 
contains another fork inside it.


- Rohini


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37906/#review97303
---


On Sept. 1, 2015, 4:46 p.m., Purshotam Shah wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/37906/
> ---
> 
> (Updated Sept. 1, 2015, 4:46 p.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-2345
> https://issues.apache.org/jira/browse/OOZIE-2345
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> We have few customers whose SLA is 8 min. They have around 30 actions. There 
> are 25 actions in fork.
> Though forked action jobs runs concurrently. forked action job submission is 
> sequential.
> Whenever NN is slow, job submission takes more time. Even if job submission 
> is delay for 30 sec. Total WF delay will be ~12 min.
> 
> 
> Diffs
> -
> 
>   core/src/main/java/org/apache/oozie/command/wf/ActionStartXCommand.java 
> e06649cf9bbc7c0997e1d4f94fa7e695533f8343 
>   core/src/main/java/org/apache/oozie/command/wf/ActionXCommand.java 
> 2616d322ea0dd85942fcb33c73d7d33af049b789 
>   
> core/src/main/java/org/apache/oozie/command/wf/ForkedActionStartXCommand.java 
> e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 
>   core/src/main/java/org/apache/oozie/command/wf/SignalXCommand.java 
> d1fcd1ac933aaeaea22cdd3fcbe55be386281fbb 
>   core/src/main/java/org/apache/oozie/service/CallableQueueService.java 
> 830a58ebe152baed0a922e78855be51d3bf1b21c 
>   core/src/main/resources/oozie-default.xml 
> 32a1df044b9910ba9cfade88005a598e0ab3a7b3 
>   
> core/src/test/java/org/apache/oozie/command/wf/TestForkedActionStartXCommand.java
>  e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 
>   core/src/test/java/org/apache/oozie/command/wf/TestReRunXCommand.java 
> 02f61668e6d6d03674f5d516853057fb94fed518 
>   core/src/test/java/org/apache/oozie/command/wf/TestSignalXCommand.java 
> 4268b301d3efff05e870b3d833ba5fd5b9cc0d5a 
> 
> Diff: https://reviews.apache.org/r/37906/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Purshotam Shah
> 
>

Re: Review Request 37906: OOZIE-2345 -Concurrent job submission for forked action

[Rohini Palaniswamy]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37906/#review97306
---

Ship it!


Ship It!

- Rohini Palaniswamy


On Sept. 1, 2015, 4:46 p.m., Purshotam Shah wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/37906/
> ---
> 
> (Updated Sept. 1, 2015, 4:46 p.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-2345
> https://issues.apache.org/jira/browse/OOZIE-2345
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> We have few customers whose SLA is 8 min. They have around 30 actions. There 
> are 25 actions in fork.
> Though forked action jobs runs concurrently. forked action job submission is 
> sequential.
> Whenever NN is slow, job submission takes more time. Even if job submission 
> is delay for 30 sec. Total WF delay will be ~12 min.
> 
> 
> Diffs
> -
> 
>   core/src/main/java/org/apache/oozie/command/wf/ActionStartXCommand.java 
> e06649cf9bbc7c0997e1d4f94fa7e695533f8343 
>   core/src/main/java/org/apache/oozie/command/wf/ActionXCommand.java 
> 2616d322ea0dd85942fcb33c73d7d33af049b789 
>   
> core/src/main/java/org/apache/oozie/command/wf/ForkedActionStartXCommand.java 
> e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 
>   core/src/main/java/org/apache/oozie/command/wf/SignalXCommand.java 
> d1fcd1ac933aaeaea22cdd3fcbe55be386281fbb 
>   core/src/main/java/org/apache/oozie/service/CallableQueueService.java 
> 830a58ebe152baed0a922e78855be51d3bf1b21c 
>   core/src/main/resources/oozie-default.xml 
> 32a1df044b9910ba9cfade88005a598e0ab3a7b3 
>   
> core/src/test/java/org/apache/oozie/command/wf/TestForkedActionStartXCommand.java
>  e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 
>   core/src/test/java/org/apache/oozie/command/wf/TestReRunXCommand.java 
> 02f61668e6d6d03674f5d516853057fb94fed518 
>   core/src/test/java/org/apache/oozie/command/wf/TestSignalXCommand.java 
> 4268b301d3efff05e870b3d833ba5fd5b9cc0d5a 
> 
> Diff: https://reviews.apache.org/r/37906/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Purshotam Shah
> 
>

Re: Review Request 37906: OOZIE-2345 -Concurrent job submission for forked action

[Rohini Palaniswamy]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37906/#review98941
---



core/src/main/java/org/apache/oozie/command/wf/SignalXCommand.java 
<https://reviews.apache.org/r/37906/#comment155705>

Can we retain this logging of exception and also add a message with the log 
statement?



core/src/main/java/org/apache/oozie/service/CallableQueueService.java (lines 
510 - 512)
<https://reviews.apache.org/r/37906/#comment155728>

There is lot of unnecessary wrapping

CallableWrapper extends PriorityDelayQueue.QueueElement> 
implements Runnable
QueueElement extends FutureTask implements Delayed
FutureTask implements RunnableFuture

CallableWrapper is already a RunnableFuture and QueueElement. Why wrap it 
again into new QueueElement. Does not make sense.  Also I don't think we need 
QueueElement extends FutureTask in the first place.

Can't CallableWrapper be just changed from Runnable to Callable and the 
expose CallableWrapper as public static class so that Fork command can directly 
use it? That will make it simple and get rid of lot of the changes.


- Rohini Palaniswamy


On Sept. 14, 2015, 8:52 p.m., Purshotam Shah wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/37906/
> ---
> 
> (Updated Sept. 14, 2015, 8:52 p.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-2345
> https://issues.apache.org/jira/browse/OOZIE-2345
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> We have few customers whose SLA is 8 min. They have around 30 actions. There 
> are 25 actions in fork.
> Though forked action jobs runs concurrently. forked action job submission is 
> sequential.
> Whenever NN is slow, job submission takes more time. Even if job submission 
> is delay for 30 sec. Total WF delay will be ~12 min.
> 
> 
> Diffs
> -
> 
>   core/src/main/java/org/apache/oozie/command/wf/ActionStartXCommand.java 
> e06649cf9bbc7c0997e1d4f94fa7e695533f8343 
>   core/src/main/java/org/apache/oozie/command/wf/ActionXCommand.java 
> 2616d322ea0dd85942fcb33c73d7d33af049b789 
>   
> core/src/main/java/org/apache/oozie/command/wf/ForkedActionStartXCommand.java 
> e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 
>   core/src/main/java/org/apache/oozie/command/wf/SignalXCommand.java 
> d1fcd1ac933aaeaea22cdd3fcbe55be386281fbb 
>   core/src/main/java/org/apache/oozie/service/CallableQueueService.java 
> 830a58ebe152baed0a922e78855be51d3bf1b21c 
>   core/src/main/java/org/apache/oozie/util/ExtendedCallableWrapper.java 
> e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 
>   core/src/main/java/org/apache/oozie/util/PriorityDelayQueue.java 
> ae545063045f70784089c27a69666ad445cb8e4e 
>   core/src/main/resources/oozie-default.xml 
> 32a1df044b9910ba9cfade88005a598e0ab3a7b3 
>   
> core/src/test/java/org/apache/oozie/command/wf/TestForkedActionStartXCommand.java
>  e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 
>   core/src/test/java/org/apache/oozie/command/wf/TestReRunXCommand.java 
> 02f61668e6d6d03674f5d516853057fb94fed518 
>   core/src/test/java/org/apache/oozie/command/wf/TestSignalXCommand.java 
> 4268b301d3efff05e870b3d833ba5fd5b9cc0d5a 
>   core/src/test/java/org/apache/oozie/util/TestPriorityDelayQueue.java 
> 857f4e37219c705b159c9f3f970fe14b090604b8 
> 
> Diff: https://reviews.apache.org/r/37906/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Purshotam Shah
> 
>

Re: Review Request 37906: OOZIE-2345 -Concurrent job submission for forked action

[Rohini Palaniswamy]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37906/#review99291
---

Ship it!


Ship It!

- Rohini Palaniswamy


On Sept. 16, 2015, 7:49 p.m., Purshotam Shah wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/37906/
> ---
> 
> (Updated Sept. 16, 2015, 7:49 p.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-2345
> https://issues.apache.org/jira/browse/OOZIE-2345
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> We have few customers whose SLA is 8 min. They have around 30 actions. There 
> are 25 actions in fork.
> Though forked action jobs runs concurrently. forked action job submission is 
> sequential.
> Whenever NN is slow, job submission takes more time. Even if job submission 
> is delay for 30 sec. Total WF delay will be ~12 min.
> 
> 
> Diffs
> -
> 
>   core/src/main/java/org/apache/oozie/command/wf/ActionStartXCommand.java 
> e06649cf9bbc7c0997e1d4f94fa7e695533f8343 
>   core/src/main/java/org/apache/oozie/command/wf/ActionXCommand.java 
> 2616d322ea0dd85942fcb33c73d7d33af049b789 
>   
> core/src/main/java/org/apache/oozie/command/wf/ForkedActionStartXCommand.java 
> e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 
>   core/src/main/java/org/apache/oozie/command/wf/SignalXCommand.java 
> d1fcd1ac933aaeaea22cdd3fcbe55be386281fbb 
>   core/src/main/java/org/apache/oozie/service/CallableQueueService.java 
> 830a58ebe152baed0a922e78855be51d3bf1b21c 
>   core/src/main/java/org/apache/oozie/util/PriorityDelayQueue.java 
> ae545063045f70784089c27a69666ad445cb8e4e 
>   core/src/main/resources/oozie-default.xml 
> 32a1df044b9910ba9cfade88005a598e0ab3a7b3 
>   
> core/src/test/java/org/apache/oozie/command/wf/TestForkedActionStartXCommand.java
>  e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 
>   core/src/test/java/org/apache/oozie/command/wf/TestReRunXCommand.java 
> 02f61668e6d6d03674f5d516853057fb94fed518 
>   core/src/test/java/org/apache/oozie/command/wf/TestSignalXCommand.java 
> 4268b301d3efff05e870b3d833ba5fd5b9cc0d5a 
>   core/src/test/java/org/apache/oozie/util/TestPriorityDelayQueue.java 
> 857f4e37219c705b159c9f3f970fe14b090604b8 
> 
> Diff: https://reviews.apache.org/r/37906/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Purshotam Shah
> 
>

Re: Review Request 39083: OOZIE-2367 : fs delete should support skipTrash option

[Rohini Palaniswamy]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39083/#review103381
---


Mostly good. Just few minor comments.


core/src/main/java/org/apache/oozie/action/hadoop/FsActionExecutor.java (line 
393)
<https://reviews.apache.org/r/39083/#comment161400>

UserGroupInformationService.getProxyUser()



core/src/main/java/org/apache/oozie/action/hadoop/FsActionExecutor.java (line 
401)
<https://reviews.apache.org/r/39083/#comment161402>

Can you change error message to

Could not move path [{0}] to trash on delete

It does not sound grammatically right to read though it conveys the 
message. I see that the text is copied from the other error message. Can you 
also just change that to

Could not delete path [{0}]



docs/src/site/twiki/WorkflowFunctionalSpec.twiki (line 2873)
<https://reviews.apache.org/r/39083/#comment161403>

xs:boolean



docs/src/site/twiki/WorkflowFunctionalSpec.twiki (line 4342)
<https://reviews.apache.org/r/39083/#comment161404>

This is schema version 0.2. Do not change here.


- Rohini Palaniswamy


On Oct. 7, 2015, 12:49 p.m., Jaydeep Vishwakarma wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/39083/
> ---
> 
> (Updated Oct. 7, 2015, 12:49 p.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-2367
> https://issues.apache.org/jira/browse/OOZIE-2367
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> providing skip trash option on fs delete, setting skip-trash to 'false' in fs 
> delete will move targeted directory to user's trash direcotry.
> 
> 
> Diffs
> -
> 
>   client/src/main/resources/oozie-workflow-0.5.xsd b01580c 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsActionExecutor.java 
> 8d96a47 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestFsActionExecutor.java 
> 86d5fa9 
>   docs/src/site/twiki/WorkflowFunctionalSpec.twiki bf4d123 
> 
> Diff: https://reviews.apache.org/r/39083/diff/
> 
> 
> Testing
> ---
> 
> yes
> 
> 
> Thanks,
> 
> Jaydeep Vishwakarma
> 
>

Re: Review Request 33207: OOZIE:2243 - Hadoop jobs doesnot get killed when coordinator action get killed

[Rohini Palaniswamy]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/33207/#review103390
---


Code is good. Testcase needs to be rewritten.


core/src/test/java/org/apache/oozie/command/wf/TestWorkflowKillXCommand.java 
(line 134)
<https://reviews.apache.org/r/33207/#comment161448>

This testcase does not actually test killing of child job. It just creates 
workflow job and action entries in database and there are no mapreduce jobs 
(launcher or child) running. Calling kill will just make the job and action 
marked killed.

You need a test like TestJavaActionExecutor.testKill(). Difference is you 
need to write a action that launches a child job and then assert if both the 
java action and child job are killed. Pig script with a Jython UDF (don't have 
to write java UDF) that does sleep for a long time is a easy one. And you will 
have to skip the test for Hadoop 1.x . i.e

org.junit.Assume.assumeTrue("Skip this test for Hadoop 1.x", 
HadoopShims.isYARN());


- Rohini Palaniswamy


On Sept. 29, 2015, 11:46 a.m., Narayan Periwal wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/33207/
> ---
> 
> (Updated Sept. 29, 2015, 11:46 a.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-2243
> https://issues.apache.org/jira/browse/OOZIE-2243
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> Whenever the coord action or workflow gets killed only the job that has error 
> gets killed while the other hadoop jobs keeps on running and only the 
> workflow status gets changed to killed.
> 
> 
> Diffs
> -
> 
>   core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 
> a6e7000 
>   core/src/main/java/org/apache/oozie/action/hadoop/LauncherMapperHelper.java 
> e22329d 
>   
> core/src/main/java/org/apache/oozie/action/oozie/SubWorkflowActionExecutor.java
>  854d621 
>   core/src/main/java/org/apache/oozie/command/wf/ActionStartXCommand.java 
> 85a6cd7 
>   
> core/src/test/java/org/apache/oozie/command/wf/TestWorkflowKillXCommand.java 
> e493d4d 
>   
> hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java
>  ce8c14f 
>   
> hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java
>  94e01ea 
> 
> Diff: https://reviews.apache.org/r/33207/diff/
> 
> 
> Testing
> ---
> 
> Done
> 
> 
> Thanks,
> 
> Narayan Periwal
> 
>

Re: Review Request 39083: OOZIE-2367 : fs delete should support skipTrash option

[Rohini Palaniswamy]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39083/#review104476
---

Ship it!


Ship It!

- Rohini Palaniswamy


On Oct. 21, 2015, 3:11 p.m., Jaydeep Vishwakarma wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/39083/
> ---
> 
> (Updated Oct. 21, 2015, 3:11 p.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-2367
> https://issues.apache.org/jira/browse/OOZIE-2367
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> providing skip trash option on fs delete, setting skip-trash to 'false' in fs 
> delete will move targeted directory to user's trash direcotry.
> 
> 
> Diffs
> -
> 
>   client/src/main/resources/oozie-workflow-0.5.xsd b01580c 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsActionExecutor.java 
> 8d96a47 
>   core/src/test/java/org/apache/oozie/action/hadoop/TestFsActionExecutor.java 
> 86d5fa9 
>   docs/src/site/twiki/WorkflowFunctionalSpec.twiki bf4d123 
> 
> Diff: https://reviews.apache.org/r/39083/diff/
> 
> 
> Testing
> ---
> 
> yes
> 
> 
> Thanks,
> 
> Jaydeep Vishwakarma
> 
>

Re: Review Request 35379: OOZIE-2030 Configuration properties from global section is not getting set in Hadoop job conf when using sub-workflow action in Oozie workflow.xml

[Rohini Palaniswamy]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/35379/#review108278
---


Like that the logic is lot more concise than the previous versions and you have 
made good use of changes in OOZIE-2187. But would like optimization to avoid 
cost of serializing GlobalSectionData for all workflows which do not even have 
sub-workflows and also the deserialization of GlobalSectionData done for every 
action again for all workflows which are unnecessary. Suggested changes below 
which will serialize global section into jobconf only when there is 
sub-workflow with propagate-configuration. Also deserialization of 
GlobalSectionData from jobconf is done only once in the beginning.


core/src/main/java/org/apache/oozie/workflow/lite/LiteWorkflowAppParser.java 
(line 431)
<https://reviews.apache.org/r/35379/#comment167645>

GlobalSectionData gData = jobConf.get(OOZIE_GLOBAL) == null ? null : 
getGlobalFromString(jobConf.get(OOZIE_GLOBAL))
boolean serializedGlobalConf = false;



core/src/main/java/org/apache/oozie/workflow/lite/LiteWorkflowAppParser.java 
(lines 474 - 477)
<https://reviews.apache.org/r/35379/#comment167632>

Instead of propagateGlobalConf = true; can we do

if (!serializedGlobalConf  && 
elem.getName().equals(SubWorkflowActionExecutor.ACTION_TYPE) &&
elem.getChild(("propagate-configuration"), 
ns) != null) {
serializedGlobalConf = true;
jobConf.set(OOZIE_GLOBAL, getGlobalString(gData));
}
eActionConf = elem;   handleDefaultsAndGlobal(gData, 
configDefault, elem); 

here instead of in eNode.getName().equals(GLOBAL) section and remove

if (!propagateGlobalConf) {
jobConf.unset(OOZIE_GLOBAL);
}

serializedGlobalConf is to avoid serializing multiple times if there is 
more than one sub-workflow which is the case most of the time.



core/src/main/java/org/apache/oozie/workflow/lite/LiteWorkflowAppParser.java 
(lines 503 - 507)
<https://reviews.apache.org/r/35379/#comment167648>

if(jobConf.get(OOZIE_GLOBAL) != null) {
 handleDefaultsAndGlobal(gData, null, eNode);
}
gData = parseGlobalSection(ns, eNode);

gData = parseGlobalSection(ns, eNode); at the beginning before if block 
needs to be removed. Parse gData only once after the if block.



core/src/main/java/org/apache/oozie/workflow/lite/LiteWorkflowAppParser.java 
(line 551)
<https://reviews.apache.org/r/35379/#comment167630>

Can you compress before base64 encoding? It is compressed before storing to 
database. But in some recent issues with hcat we found that conf compresses lot 
better before base64 encoding than after it. So compressing here should also be 
good.

Deflater def = new Deflater();
oos = new DataOutputStream(new DeflaterOutputStream(baos, def));



core/src/main/java/org/apache/oozie/workflow/lite/LiteWorkflowAppParser.java 
(lines 643 - 646)
<https://reviews.apache.org/r/35379/#comment167629>

Make it an empty constructor. null is the default value with the variable 
declaration. If you want it to be explicitly null, assign it to null in the 
variable declaration itself



core/src/test/java/org/apache/oozie/action/oozie/TestSubWorkflowActionExecutor.java
 (line 709)
<https://reviews.apache.org/r/35379/#comment167631>

Shouldn't foo3 be actionconf here also?


- Rohini Palaniswamy


On July 21, 2015, 10:59 a.m., Jaydeep Vishwakarma wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/35379/
> ---
> 
> (Updated July 21, 2015, 10:59 a.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-2030
> https://issues.apache.org/jira/browse/OOZIE-2030
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> Forwarding global conf from workflow to action when action is subworkflow, 
> subworkflow storing it to property and forwarding as property. 
>  LiteworkflowAppParser handling all condition and ensuring child level pass 
> of global conf.
> 
> 
> Diffs
> -
> 
>   
> core/src/main/java/org/apache/oozie/action/oozie/SubWorkflowActionExecutor.java
>  854d621 
>   
> core/src/main/java/org/apache/oozie/workflow/lite/LiteWorkflowAppParser.java 
> d3a6523 
>   
> core/src/test/java/org/apache/oozie/action/oozie/TestSubWorkflowActionExecutor.java
>  9ab897a 
> 
> Diff: https://reviews.apache.org/r/35379/diff/
> 
> 
> Testing
> ---
> 
> yes
> 
> 
> Thanks,
> 
> Jaydeep Vishwakarma
> 
>

Re: Review Request 35379: OOZIE-2030 Configuration properties from global section is not getting set in Hadoop job conf when using sub-workflow action in Oozie workflow.xml

[Rohini Palaniswamy]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/35379/#review108386
---



core/src/main/java/org/apache/oozie/workflow/lite/LiteWorkflowAppParser.java 
(line 506)
<https://reviews.apache.org/r/35379/#comment167828>

This line should be removed.



core/src/main/java/org/apache/oozie/workflow/lite/LiteWorkflowAppParser.java 
(lines 518 - 520)
<https://reviews.apache.org/r/35379/#comment167827>

This block is not required as it is being set only when 
propagate-configuration is true.



core/src/main/java/org/apache/oozie/workflow/lite/LiteWorkflowAppParser.java 
(line 109)
<https://reviews.apache.org/r/35379/#comment167831>

oozie.wf.globalconf



core/src/main/java/org/apache/oozie/workflow/lite/LiteWorkflowAppParser.java 
(line 508)
<https://reviews.apache.org/r/35379/#comment167834>

if(jobConf.get(OOZIE_GLOBAL) != null) {
gData = getGlobalFromString(jobConf.get(OOZIE_GLOBAL));
 handleDefaultsAndGlobal(gData, null, eNode);
 }



core/src/main/java/org/apache/oozie/workflow/lite/LiteWorkflowAppParser.java 
(line 673)
<https://reviews.apache.org/r/35379/#comment167833>

writeUTF(null)



core/src/main/java/org/apache/oozie/workflow/lite/LiteWorkflowAppParser.java 
(lines 736 - 739)
<https://reviews.apache.org/r/35379/#comment167832>

This needs to be removed and GlobalSectionData gData passed as method 
parameter to avoid deserializing for every action.


- Rohini Palaniswamy


On Nov. 30, 2015, 7:50 p.m., Jaydeep Vishwakarma wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/35379/
> ---
> 
> (Updated Nov. 30, 2015, 7:50 p.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-2030
> https://issues.apache.org/jira/browse/OOZIE-2030
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> Forwarding global conf from workflow to action when action is subworkflow, 
> subworkflow storing it to property and forwarding as property. 
>  LiteworkflowAppParser handling all condition and ensuring child level pass 
> of global conf.
> 
> 
> Diffs
> -
> 
>   
> core/src/main/java/org/apache/oozie/action/oozie/SubWorkflowActionExecutor.java
>  854d621 
>   
> core/src/main/java/org/apache/oozie/workflow/lite/LiteWorkflowAppParser.java 
> d3a6523 
>   
> core/src/test/java/org/apache/oozie/action/oozie/TestSubWorkflowActionExecutor.java
>  9ab897a 
> 
> Diff: https://reviews.apache.org/r/35379/diff/
> 
> 
> Testing
> ---
> 
> yes
> 
> 
> Thanks,
> 
> Jaydeep Vishwakarma
> 
>

Re: Review Request 35379: OOZIE-2030 Configuration properties from global section is not getting set in Hadoop job conf when using sub-workflow action in Oozie workflow.xml

[Rohini Palaniswamy]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/35379/#review108633
---



core/src/main/java/org/apache/oozie/workflow/lite/LiteWorkflowAppParser.java 
(line 511)
<https://reviews.apache.org/r/35379/#comment168088>

This line needs to be removed. Purpose of my ask to make all these changes 
is to avoid cost of serializing global conf if there is no sub-workflow with 
propagate-configuration.


- Rohini Palaniswamy


On Nov. 30, 2015, 8:59 p.m., Jaydeep Vishwakarma wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/35379/
> ---
> 
> (Updated Nov. 30, 2015, 8:59 p.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-2030
> https://issues.apache.org/jira/browse/OOZIE-2030
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> Forwarding global conf from workflow to action when action is subworkflow, 
> subworkflow storing it to property and forwarding as property. 
>  LiteworkflowAppParser handling all condition and ensuring child level pass 
> of global conf.
> 
> 
> Diffs
> -
> 
>   
> core/src/main/java/org/apache/oozie/action/oozie/SubWorkflowActionExecutor.java
>  854d621 
>   
> core/src/main/java/org/apache/oozie/workflow/lite/LiteWorkflowAppParser.java 
> d3a6523 
>   
> core/src/test/java/org/apache/oozie/action/oozie/TestSubWorkflowActionExecutor.java
>  9ab897a 
> 
> Diff: https://reviews.apache.org/r/35379/diff/
> 
> 
> Testing
> ---
> 
> yes
> 
> 
> Thanks,
> 
> Jaydeep Vishwakarma
> 
>

Re: Review Request 35379: OOZIE-2030 Configuration properties from global section is not getting set in Hadoop job conf when using sub-workflow action in Oozie workflow.xml

[Rohini Palaniswamy]

> On Dec. 3, 2015, 4:02 p.m., Rohini Palaniswamy wrote:
> > core/src/main/java/org/apache/oozie/workflow/lite/LiteWorkflowAppParser.java,
> >  line 512
> > <https://reviews.apache.org/r/35379/diff/9/?file=1149310#file1149310line512>
> >
> > This line needs to be removed. Purpose of my ask to make all these 
> > changes is to avoid cost of serializing global conf if there is no 
> > sub-workflow with propagate-configuration.

This issue is still there


- Rohini


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/35379/#review108633
---


On Nov. 30, 2015, 8:59 p.m., Jaydeep Vishwakarma wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/35379/
> ---
> 
> (Updated Nov. 30, 2015, 8:59 p.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-2030
> https://issues.apache.org/jira/browse/OOZIE-2030
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> Forwarding global conf from workflow to action when action is subworkflow, 
> subworkflow storing it to property and forwarding as property. 
>  LiteworkflowAppParser handling all condition and ensuring child level pass 
> of global conf.
> 
> 
> Diffs
> -
> 
>   
> core/src/main/java/org/apache/oozie/action/oozie/SubWorkflowActionExecutor.java
>  854d621 
>   
> core/src/main/java/org/apache/oozie/workflow/lite/LiteWorkflowAppParser.java 
> d3a6523 
>   
> core/src/test/java/org/apache/oozie/action/oozie/TestSubWorkflowActionExecutor.java
>  9ab897a 
> 
> Diff: https://reviews.apache.org/r/35379/diff/
> 
> 
> Testing
> ---
> 
> yes
> 
> 
> Thanks,
> 
> Jaydeep Vishwakarma
> 
>

Re: Review Request 35379: OOZIE-2030 Configuration properties from global section is not getting set in Hadoop job conf when using sub-workflow action in Oozie workflow.xml

[Rohini Palaniswamy]

> On Dec. 3, 2015, 4:02 p.m., Rohini Palaniswamy wrote:
> > core/src/main/java/org/apache/oozie/workflow/lite/LiteWorkflowAppParser.java,
> >  line 512
> > <https://reviews.apache.org/r/35379/diff/9/?file=1149310#file1149310line512>
> >
> > This line needs to be removed. Purpose of my ask to make all these 
> > changes is to avoid cost of serializing global conf if there is no 
> > sub-workflow with propagate-configuration.
> 
> Rohini Palaniswamy wrote:
> This issue is still there

I reopened the issue on the same day and also commented, but forgot to actually 
publish the comment and it stayed as a draft. Was wondering why you hadn't 
replied.  Really sorry about that. 

jobConf.set(OOZIE_GLOBAL, getGlobalString(gData)); is still not removed in the 
final patch and that needs to be done.


- Rohini


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/35379/#review108633
---


On Nov. 30, 2015, 8:59 p.m., Jaydeep Vishwakarma wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/35379/
> ---
> 
> (Updated Nov. 30, 2015, 8:59 p.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-2030
> https://issues.apache.org/jira/browse/OOZIE-2030
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> Forwarding global conf from workflow to action when action is subworkflow, 
> subworkflow storing it to property and forwarding as property. 
>  LiteworkflowAppParser handling all condition and ensuring child level pass 
> of global conf.
> 
> 
> Diffs
> -
> 
>   
> core/src/main/java/org/apache/oozie/action/oozie/SubWorkflowActionExecutor.java
>  854d621 
>   
> core/src/main/java/org/apache/oozie/workflow/lite/LiteWorkflowAppParser.java 
> d3a6523 
>   
> core/src/test/java/org/apache/oozie/action/oozie/TestSubWorkflowActionExecutor.java
>  9ab897a 
> 
> Diff: https://reviews.apache.org/r/35379/diff/
> 
> 
> Testing
> ---
> 
> yes
> 
> 
> Thanks,
> 
> Jaydeep Vishwakarma
> 
>

Re: Review Request 35379: OOZIE-2030 Configuration properties from global section is not getting set in Hadoop job conf when using sub-workflow action in Oozie workflow.xml

[Rohini Palaniswamy]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/35379/#review111533
---

Ship it!


Ship It!

- Rohini Palaniswamy


On Dec. 16, 2015, 10:23 a.m., Jaydeep Vishwakarma wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/35379/
> ---
> 
> (Updated Dec. 16, 2015, 10:23 a.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-2030
> https://issues.apache.org/jira/browse/OOZIE-2030
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> Forwarding global conf from workflow to action when action is subworkflow, 
> subworkflow storing it to property and forwarding as property. 
>  LiteworkflowAppParser handling all condition and ensuring child level pass 
> of global conf.
> 
> 
> Diffs
> -
> 
>   
> core/src/main/java/org/apache/oozie/action/oozie/SubWorkflowActionExecutor.java
>  854d621 
>   
> core/src/main/java/org/apache/oozie/workflow/lite/LiteWorkflowAppParser.java 
> d3a6523 
>   
> core/src/test/java/org/apache/oozie/action/oozie/TestSubWorkflowActionExecutor.java
>  9ab897a 
> 
> Diff: https://reviews.apache.org/r/35379/diff/
> 
> 
> Testing
> ---
> 
> yes
> 
> 
> Thanks,
> 
> Jaydeep Vishwakarma
> 
>

Re: Review Request 35379: OOZIE-2030 Configuration properties from global section is not getting set in Hadoop job conf when using sub-workflow action in Oozie workflow.xml

[Rohini Palaniswamy]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/35379/#review111534
---



core/src/main/java/org/apache/oozie/workflow/lite/LiteWorkflowAppParser.java 
<https://reviews.apache.org/r/35379/#comment171727>

You had removed it from here and that should not be the case. This needs to 
be there so that it is only set when there is sub-workflow and propagate conf.



core/src/main/java/org/apache/oozie/workflow/lite/LiteWorkflowAppParser.java 
(line 510)
<https://reviews.apache.org/r/35379/#comment171728>

This is where it needs to be removed so that it is not set for every 
workflow having global section


- Rohini Palaniswamy


On Dec. 16, 2015, 10:23 a.m., Jaydeep Vishwakarma wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/35379/
> ---
> 
> (Updated Dec. 16, 2015, 10:23 a.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-2030
> https://issues.apache.org/jira/browse/OOZIE-2030
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> Forwarding global conf from workflow to action when action is subworkflow, 
> subworkflow storing it to property and forwarding as property. 
>  LiteworkflowAppParser handling all condition and ensuring child level pass 
> of global conf.
> 
> 
> Diffs
> -
> 
>   
> core/src/main/java/org/apache/oozie/action/oozie/SubWorkflowActionExecutor.java
>  854d621 
>   
> core/src/main/java/org/apache/oozie/workflow/lite/LiteWorkflowAppParser.java 
> d3a6523 
>   
> core/src/test/java/org/apache/oozie/action/oozie/TestSubWorkflowActionExecutor.java
>  9ab897a 
> 
> Diff: https://reviews.apache.org/r/35379/diff/
> 
> 
> Testing
> ---
> 
> yes
> 
> 
> Thanks,
> 
> Jaydeep Vishwakarma
> 
>

Re: Review Request 35379: OOZIE-2030 Configuration properties from global section is not getting set in Hadoop job conf when using sub-workflow action in Oozie workflow.xml

[Rohini Palaniswamy]

> On Dec. 21, 2015, 9:12 p.m., Rohini Palaniswamy wrote:
> > Ship It!

Ignore this one. Requested change was in the wrong place. Updated comments 
below.


- Rohini


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/35379/#review111533
---


On Dec. 16, 2015, 10:23 a.m., Jaydeep Vishwakarma wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/35379/
> ---
> 
> (Updated Dec. 16, 2015, 10:23 a.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-2030
> https://issues.apache.org/jira/browse/OOZIE-2030
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> Forwarding global conf from workflow to action when action is subworkflow, 
> subworkflow storing it to property and forwarding as property. 
>  LiteworkflowAppParser handling all condition and ensuring child level pass 
> of global conf.
> 
> 
> Diffs
> -
> 
>   
> core/src/main/java/org/apache/oozie/action/oozie/SubWorkflowActionExecutor.java
>  854d621 
>   
> core/src/main/java/org/apache/oozie/workflow/lite/LiteWorkflowAppParser.java 
> d3a6523 
>   
> core/src/test/java/org/apache/oozie/action/oozie/TestSubWorkflowActionExecutor.java
>  9ab897a 
> 
> Diff: https://reviews.apache.org/r/35379/diff/
> 
> 
> Testing
> ---
> 
> yes
> 
> 
> Thanks,
> 
> Jaydeep Vishwakarma
> 
>

Re: Review Request 35379: OOZIE-2030 Configuration properties from global section is not getting set in Hadoop job conf when using sub-workflow action in Oozie workflow.xml

[Rohini Palaniswamy]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/35379/#review111618
---



core/src/main/java/org/apache/oozie/workflow/lite/LiteWorkflowAppParser.java 
(line 669)
<https://reviews.apache.org/r/35379/#comment171826>

One last minor thing - WritableUtils.writeStr(dataOutput, null) . Not sure 
how I missed this earlier.


- Rohini Palaniswamy


On Dec. 22, 2015, 6:49 a.m., Jaydeep Vishwakarma wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/35379/
> ---
> 
> (Updated Dec. 22, 2015, 6:49 a.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-2030
> https://issues.apache.org/jira/browse/OOZIE-2030
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> Forwarding global conf from workflow to action when action is subworkflow, 
> subworkflow storing it to property and forwarding as property. 
>  LiteworkflowAppParser handling all condition and ensuring child level pass 
> of global conf.
> 
> 
> Diffs
> -
> 
>   
> core/src/main/java/org/apache/oozie/action/oozie/SubWorkflowActionExecutor.java
>  854d621 
>   
> core/src/main/java/org/apache/oozie/workflow/lite/LiteWorkflowAppParser.java 
> d3a6523 
>   
> core/src/test/java/org/apache/oozie/action/oozie/TestSubWorkflowActionExecutor.java
>  9ab897a 
> 
> Diff: https://reviews.apache.org/r/35379/diff/
> 
> 
> Testing
> ---
> 
> yes
> 
> 
> Thanks,
> 
> Jaydeep Vishwakarma
> 
>

Re: Review Request 41623: OOZIE-1922 - MemoryLocksService fails if lock is acquired multiple times in same thread and released

[Rohini Palaniswamy]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41623/#review111682
---



core/src/main/java/org/apache/oozie/service/ZKLocksService.java (line 201)
<https://reviews.apache.org/r/41623/#comment171905>

readWriteLock != null - Instead of adding this condition in multiple 
places, either put the whole logic in 

if (readWriteLock != null) {

}

block or make the InterProcessReadWriteLock also part of ZKLockToken like 
MemoryLockToken so that you don't have to retrieve from hashmap.



core/src/test/java/org/apache/oozie/service/TestZKLocksService.java (line 484)
<https://reviews.apache.org/r/41623/#comment171907>

Test case is most likely flaky. It needs to be changed to make the other 
thread acquire the lock after the thread that acquired the lock first released 
it. With current logic mostly only one thread will acquire the lock and release 
before the assert statement or if the other thread acquired the lock during 
sleep(1000), then test will fail. 

Also need a new testcase testReentrantSameThread() - to test reentrant 
behaviour in same thread.


- Rohini Palaniswamy


On Dec. 22, 2015, 1:11 a.m., Purshotam Shah wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/41623/
> ---
> 
> (Updated Dec. 22, 2015, 1:11 a.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-1922
> https://issues.apache.org/jira/browse/OOZIE-1922
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> OOZIE-1922 - MemoryLocksService fails if lock is acquired multiple times in 
> same thread and released
> 
> 
> Diffs
> -
> 
>   core/src/main/java/org/apache/oozie/lock/MemoryLocks.java 
> ee564b3def2ce42b06f697c3384b1e102ac6a3ba 
>   core/src/main/java/org/apache/oozie/service/MemoryLocksService.java 
> e3eccdb3f95e74198c3d42d19022b4acfc3d18e5 
>   core/src/main/java/org/apache/oozie/service/ZKLocksService.java 
> e3a6bcf04048e140d2ced9d8bdbfe08b01b323e4 
>   core/src/test/java/org/apache/oozie/lock/TestMemoryLocks.java 
> 0efe31033f63055c08e42202c56c336248271afe 
>   core/src/test/java/org/apache/oozie/service/TestZKLocksService.java 
> 02cc1372dc015713f0bb1f6861b7e4b7455c4f13 
> 
> Diff: https://reviews.apache.org/r/41623/diff/
> 
> 
> Testing
> ---
> 
> TestZKLocksService.testReentrantMultipleCall already has reentrant test cases.
> 
> 
> Thanks,
> 
> Purshotam Shah
> 
>

Re: Review Request 39144: OOZIE-2312 oozie doesn't purge audit and error log

[Rohini Palaniswamy]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39144/#review111685
---



core/src/main/conf/oozie-log4j.properties (line 90)
<https://reviews.apache.org/r/39144/#comment171917>

We seem to be going from '.'-MM-dd to --MM-dd. Old files will be 
ignored due to pattern change. That is fine as it will only be for max days and 
not frequently accessed by users.

But will new pattern be picked up? I don't see any code changes related to 
that. Can you check Oozie Audit Log tab in UI?

XLogUtil.extractInfoForLogWebService() {

else if 
(appenderClass.equals("org.apache.log4j.rolling.RollingFileAppender")) {
...
log.warn(
"Oozie WS "
+ logType
+ " log will be disabled, 
RollingPolicy.FileNamePattern [{0}] should end with "
+ "'-%d{-MM-dd-HH}' or 
'-%d{-MM-dd-HH}.gz' and also start with the value of "
+ "log4j.appender." + logType + 
".File [{1}]", pattern, logFile);
..
}
}

-%d{-MM-dd} is not recognized there and it will disable audit logging 
as far as I see.



core/src/main/java/org/apache/oozie/util/OozieRollingPolicy.java (lines 57 - 58)
<https://reviews.apache.org/r/39144/#comment171913>

Can you call the variables oozieLogDir and logFileName? Makes it easy to 
understand that the first one is the directory and not the full log file path 
and second one is just the name.



core/src/main/java/org/apache/oozie/util/OozieRollingPolicy.java (line 174)
<https://reviews.apache.org/r/39144/#comment171911>

This logic is simple and good considering only two formats (day and hour) 
are supported. Can you create a new jira to cleanup 
XLogStreamer.getGZFileCreationTime as well? Do mention in the jira that we need 
to make SimpleDateFormat as a thread local variable in one class to be used by 
both the methods.



core/src/test/java/org/apache/oozie/util/TestOozieRollingPolicy.java (line 62)
<https://reviews.apache.org/r/39144/#comment171914>

Typo. Should be calendarUnit



core/src/test/java/org/apache/oozie/util/TestOozieRollingPolicy.java (line 85)
<https://reviews.apache.org/r/39144/#comment171916>

calendarUnit == Calendar.HOUR_OF_DAY instead of isAuditLog to determine 
isHourlyLog.



core/src/test/java/org/apache/oozie/util/TestOozieRollingPolicy.java (line 152)
<https://reviews.apache.org/r/39144/#comment171915>

To make it generic, can you change to

boolean isAuditLog- > boolean isHourlyLog

if (!isAuditLog) -> if (isHourlyLog)


- Rohini Palaniswamy


On Oct. 8, 2015, 8:53 p.m., Purshotam Shah wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/39144/
> ---
> 
> (Updated Oct. 8, 2015, 8:53 p.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-2312
> https://issues.apache.org/jira/browse/OOZIE-2312
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> oozie doesn't purge audit and error log
> 
> 
> Diffs
> -
> 
>   core/src/main/conf/oozie-log4j.properties 
> 2b20ff282477d91549e3f6cdee425674e4cc773b 
>   core/src/main/java/org/apache/oozie/util/OozieRollingPolicy.java 
> 19ce7f95d3914d006f257e8ad4cc9b68e1d6488c 
>   core/src/test/java/org/apache/oozie/util/TestOozieRollingPolicy.java 
> 9ae7bec99fa9b47ca05a485c2f47bb3f642a54a2 
> 
> Diff: https://reviews.apache.org/r/39144/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Purshotam Shah
> 
>

Fwd: [jira] Subscription: Oozie Patch Available

[Rohini Palaniswamy]

Committers,
I see that patches submitted by users have been piling up and we are at
57 now. It would be good to get that down to 20 and avoid patches going too
stale and contributors disappearing due to lack of response on our end. If
everyone can pick up 3-5 jiras for reviewing and commiting it can be
cleared up quickly. If the patches are not relevant, clear the Patch
available or close them so that it gets removed from this report.

   If there are contributors interested in reviewing, please go ahead and
pick some patches for reviewing. It would make it easy for the committers
and also help one move towards committership.

Regards,
Rohini


-- Forwarded message --
From: 
Date: Sun, Dec 27, 2015 at 11:00 PM
Subject: [jira] Subscription: Oozie Patch Available
To: dev@oozie.apache.org


Issue Subscription
Filter: Oozie Patch Available (57 issues)

Subscriber: ooziedaily

Key Summary
OOZIE-2419  HBase credentials are not correctly proxied
https://issues.apache.org/jira/browse/OOZIE-2419
OOZIE-2409  hive2 action with hive 1.2.1 failed
https://issues.apache.org/jira/browse/OOZIE-2409
OOZIE-2400  Workflow xml configuration parser cannot deal with namespace
prefix
https://issues.apache.org/jira/browse/OOZIE-2400
OOZIE-2396  oozie compile with hadoop 2.7.1 failed
https://issues.apache.org/jira/browse/OOZIE-2396
OOZIE-2394  Oozie can execute command without holding lock
https://issues.apache.org/jira/browse/OOZIE-2394
OOZIE-2390  Rerun with failed option removing completed output data
https://issues.apache.org/jira/browse/OOZIE-2390
OOZIE-2362  SQL injection in BulkJPAExecutor
https://issues.apache.org/jira/browse/OOZIE-2362
OOZIE-2357  CoordRerun should queue CoordPushDependencyCheckXCommand if
there is push dependencies
https://issues.apache.org/jira/browse/OOZIE-2357
OOZIE-2349  Method getCoordJobInfo(String jobId, String filter, int offset,
int length, boolean desc) is not present in LocalOozieClientCoord
https://issues.apache.org/jira/browse/OOZIE-2349
OOZIE-2338  Invalid configuration defined reported for some valid configs
https://issues.apache.org/jira/browse/OOZIE-2338
OOZIE-2312  oozie doesn't purge audit and error log
https://issues.apache.org/jira/browse/OOZIE-2312
OOZIE-2273  MiniOozie does not work outside of Oozie
https://issues.apache.org/jira/browse/OOZIE-2273
OOZIE-2259  Create a callback action
https://issues.apache.org/jira/browse/OOZIE-2259
OOZIE-2258  Introducing a new counter in the instrumentation log to
distinguish between the reasons for launcher failure
https://issues.apache.org/jira/browse/OOZIE-2258
OOZIE-2253  Spark Job is failing when it is running in standalone server
https://issues.apache.org/jira/browse/OOZIE-2253
OOZIE-2244  Oozie should mask passwords in the logs when logging command
arguments
https://issues.apache.org/jira/browse/OOZIE-2244
OOZIE-2243  Kill Command does not kill the child job for java action
https://issues.apache.org/jira/browse/OOZIE-2243
OOZIE-2203  Fix the login example
https://issues.apache.org/jira/browse/OOZIE-2203
OOZIE-2196  Create Local Client for Bundle
https://issues.apache.org/jira/browse/OOZIE-2196
OOZIE-2172  ZooKeeper Security Tests failed with JVM IBM JAVA
https://issues.apache.org/jira/browse/OOZIE-2172
OOZIE-2134  Remove references to Services.get().getConf() in code
https://issues.apache.org/jira/browse/OOZIE-2134
OOZIE-2106  Make tomcat download url configurable in the pom file
https://issues.apache.org/jira/browse/OOZIE-2106
OOZIE-2105  Make version of submodules configurable with parent version
https://issues.apache.org/jira/browse/OOZIE-2105
OOZIE-2099  Add test-patch support for patches generated without --no-prefix
https://issues.apache.org/jira/browse/OOZIE-2099
OOZIE-2081  WorkflowJob notification to include coordinator action id
https://issues.apache.org/jira/browse/OOZIE-2081
OOZIE-2060  Incorrect documentation of Java action config XML filename
https://issues.apache.org/jira/browse/OOZIE-2060
OOZIE-2044  ssh action succeed with a not exists command which should be
fail.
https://issues.apache.org/jira/browse/OOZIE-2044
OOZIE-2030  Configuration properties from global section is not getting set
in Hadoop job conf when using sub-workflow action in Oozie workflow.xml
https://issues.apache.org/jira/browse/OOZIE-2030
OOZIE-2020  Rerun all Failed/killed/timedout coordinator actions rather
than specifying action numbers
https://issues.apache.org/jira/browse/OOZIE-2020
OOZIE-1980  Sql error should not fail coord job
https://issues.apache.org/jira/browse/OOZIE-1980
OOZIE-1977  Display patch analysis issues
https://issues.apache.org/jira/browse/OOZ

Reviewing patches and clearing backlog

[Rohini Palaniswamy]

Committers,
I see that patches submitted by users have been piling up and we are at
57 now. It would be good to get that down to 20 and avoid patches going too
stale and contributors disappearing due to lack of response on our end. If
everyone can pick up 3-5 jiras for reviewing and commiting it can be
cleared up quickly. If the patches are not relevant, clear the Patch
available or close them so that it gets removed from this report.

   If there are contributors interested in reviewing, please go ahead and
pick some patches for reviewing. It would make it easy for the committers
and also help one move towards committership.

Thanks,
Rohini


-- Forwarded message --
From: 
Date: Sun, Dec 27, 2015 at 11:00 PM
Subject: [jira] Subscription: Oozie Patch Available
To: dev@oozie.apache.org


Issue Subscription
Filter: Oozie Patch Available (57 issues)

Subscriber: ooziedaily

Key Summary
OOZIE-2419  HBase credentials are not correctly proxied
https://issues.apache.org/jira/browse/OOZIE-2419
OOZIE-2409  hive2 action with hive 1.2.1 failed
https://issues.apache.org/jira/browse/OOZIE-2409
OOZIE-2400  Workflow xml configuration parser cannot deal with namespace
prefix
https://issues.apache.org/jira/browse/OOZIE-2400
OOZIE-2396  oozie compile with hadoop 2.7.1 failed
https://issues.apache.org/jira/browse/OOZIE-2396
OOZIE-2394  Oozie can execute command without holding lock
https://issues.apache.org/jira/browse/OOZIE-2394
OOZIE-2390  Rerun with failed option removing completed output data
https://issues.apache.org/jira/browse/OOZIE-2390
OOZIE-2362  SQL injection in BulkJPAExecutor
https://issues.apache.org/jira/browse/OOZIE-2362
OOZIE-2357  CoordRerun should queue CoordPushDependencyCheckXCommand if
there is push dependencies
https://issues.apache.org/jira/browse/OOZIE-2357
OOZIE-2349  Method getCoordJobInfo(String jobId, String filter, int offset,
int length, boolean desc) is not present in LocalOozieClientCoord
https://issues.apache.org/jira/browse/OOZIE-2349
OOZIE-2338  Invalid configuration defined reported for some valid configs
https://issues.apache.org/jira/browse/OOZIE-2338
OOZIE-2312  oozie doesn't purge audit and error log
https://issues.apache.org/jira/browse/OOZIE-2312
OOZIE-2273  MiniOozie does not work outside of Oozie
https://issues.apache.org/jira/browse/OOZIE-2273
OOZIE-2259  Create a callback action
https://issues.apache.org/jira/browse/OOZIE-2259
OOZIE-2258  Introducing a new counter in the instrumentation log to
distinguish between the reasons for launcher failure
https://issues.apache.org/jira/browse/OOZIE-2258
OOZIE-2253  Spark Job is failing when it is running in standalone server
https://issues.apache.org/jira/browse/OOZIE-2253
OOZIE-2244  Oozie should mask passwords in the logs when logging command
arguments
https://issues.apache.org/jira/browse/OOZIE-2244
OOZIE-2243  Kill Command does not kill the child job for java action
https://issues.apache.org/jira/browse/OOZIE-2243
OOZIE-2203  Fix the login example
https://issues.apache.org/jira/browse/OOZIE-2203
OOZIE-2196  Create Local Client for Bundle
https://issues.apache.org/jira/browse/OOZIE-2196
OOZIE-2172  ZooKeeper Security Tests failed with JVM IBM JAVA
https://issues.apache.org/jira/browse/OOZIE-2172
OOZIE-2134  Remove references to Services.get().getConf() in code
https://issues.apache.org/jira/browse/OOZIE-2134
OOZIE-2106  Make tomcat download url configurable in the pom file
https://issues.apache.org/jira/browse/OOZIE-2106
OOZIE-2105  Make version of submodules configurable with parent version
https://issues.apache.org/jira/browse/OOZIE-2105
OOZIE-2099  Add test-patch support for patches generated without --no-prefix
https://issues.apache.org/jira/browse/OOZIE-2099
OOZIE-2081  WorkflowJob notification to include coordinator action id
https://issues.apache.org/jira/browse/OOZIE-2081
OOZIE-2060  Incorrect documentation of Java action config XML filename
https://issues.apache.org/jira/browse/OOZIE-2060
OOZIE-2044  ssh action succeed with a not exists command which should be
fail.
https://issues.apache.org/jira/browse/OOZIE-2044
OOZIE-2030  Configuration properties from global section is not getting set
in Hadoop job conf when using sub-workflow action in Oozie workflow.xml
https://issues.apache.org/jira/browse/OOZIE-2030
OOZIE-2020  Rerun all Failed/killed/timedout coordinator actions rather
than specifying action numbers
https://issues.apache.org/jira/browse/OOZIE-2020
OOZIE-1980  Sql error should not fail coord job
https://issues.apache.org/jira/browse/OOZIE-1980
OOZIE-1977  Display patch analysis issues
https://issues.apache.org/jira/browse/OOZI

Re: Review Request 41623: OOZIE-1922 - MemoryLocksService fails if lock is acquired multiple times in same thread and released

[Rohini Palaniswamy]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41623/#review112195
---



core/src/main/java/org/apache/oozie/service/ZKLocksService.java (line 200)
<https://reviews.apache.org/r/41623/#comment172521>

The null check should be here instead of isLockHeld.

InterProcessReadWriteLock readWriteLock = zkLocks.get(resource);
//It should never be null, but just in case
if (readWriteLock != null) { 
  if (!isLockHeld(readWriteLock)) {
  
  }
}


- Rohini Palaniswamy


On Dec. 29, 2015, 12:31 a.m., Purshotam Shah wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/41623/
> ---
> 
> (Updated Dec. 29, 2015, 12:31 a.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-1922
> https://issues.apache.org/jira/browse/OOZIE-1922
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> OOZIE-1922 - MemoryLocksService fails if lock is acquired multiple times in 
> same thread and released
> 
> 
> Diffs
> -
> 
>   core/src/main/java/org/apache/oozie/lock/MemoryLocks.java 
> ee564b3def2ce42b06f697c3384b1e102ac6a3ba 
>   core/src/main/java/org/apache/oozie/service/MemoryLocksService.java 
> e3eccdb3f95e74198c3d42d19022b4acfc3d18e5 
>   core/src/main/java/org/apache/oozie/service/ZKLocksService.java 
> e3a6bcf04048e140d2ced9d8bdbfe08b01b323e4 
>   core/src/test/java/org/apache/oozie/lock/TestMemoryLocks.java 
> 0efe31033f63055c08e42202c56c336248271afe 
>   core/src/test/java/org/apache/oozie/service/TestZKLocksService.java 
> 02cc1372dc015713f0bb1f6861b7e4b7455c4f13 
> 
> Diff: https://reviews.apache.org/r/41623/diff/
> 
> 
> Testing
> ---
> 
> TestZKLocksService.testReentrantMultipleCall already has reentrant test cases.
> 
> 
> Thanks,
> 
> Purshotam Shah
> 
>

Re: Review Request 38474: OOZIE-1976- Specifying coordinator input datasets in more logical ways

[Rohini Palaniswamy]

dInputDependency.java
 (line 64)
<https://reviews.apache.org/r/38474/#comment172530>

Nitpick. Does nothing and can be removed



core/src/main/java/org/apache/oozie/coord/input/dependency/AbstractCoordInputDependency.java
 (lines 299 - 300)
<https://reviews.apache.org/r/38474/#comment172548>

Can write the boolean first and then map. Can stop deserializing if 
dependency is met for display purposes.



core/src/main/java/org/apache/oozie/coord/input/dependency/CoordInputDependency.java
 (line 33)
<https://reviews.apache.org/r/38474/#comment172519>

V=1



core/src/main/java/org/apache/oozie/coord/input/dependency/CoordInputDependency.java
 (lines 96 - 98)
<https://reviews.apache.org/r/38474/#comment172547>

join -> missingDependencies



core/src/main/java/org/apache/oozie/coord/input/dependency/CoordInputDependencyFactory.java
 (line 43)
<https://reviews.apache.org/r/38474/#comment172554>

isInputCheck -> hasInputLogic. Other places too



core/src/main/java/org/apache/oozie/coord/input/dependency/CoordInputDependencyFactory.java
 (line 58)
<https://reviews.apache.org/r/38474/#comment172555>

Nitpick. Pushinput -> PushInput



core/src/main/java/org/apache/oozie/util/WritableUtils.java (line 130)
<https://reviews.apache.org/r/38474/#comment172517>

readStringList



core/src/main/java/org/apache/oozie/util/WritableUtils.java (line 154)
<https://reviews.apache.org/r/38474/#comment172518>

writeStringList


- Rohini Palaniswamy


On Dec. 15, 2015, 7:33 p.m., Purshotam Shah wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/38474/
> ---
> 
> (Updated Dec. 15, 2015, 7:33 p.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-1976
> https://issues.apache.org/jira/browse/OOZIE-1976
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> There are three components in this patch
> 
> 1. User interface
> A new tag is added to coordinator.xml
> ex.
> 
> 
>   
>   "
>   
>
>
>   
>   
>
>
>  ;
> 
> 
> 
> input-check will have nested and/or/combine operation. It can have min and 
> wait at operator or at date-in.
> If input-check tag is missing then it consider to be old approach where all 
> data dependency are needed.
> 
> 2. Processing
> input-check is converted into logical expression
>   (a&&B)||(c&&d)
> We use jexl to parse the logical expression.
> 
> There are three phase in parsing.
> phase 1 : only resolved dataset are parsed ( only current).   
> phase 2 : once all current are resolved, then future/latest are parsed.
> phase 3 : Doesn't do any filecheck, just return what is being parsed by 
> phase1 and phase2. Is used for EL functions
> 
> 
> 3. Storage.
> if inputcheck is enable, push_missing_dependencies and missing_dependencies 
> are serialized and stored in DB.
> If then not then it's old approach, where they are stored in plan text. This 
> is backward compatible.
> 
> 
> Diffs
> -
> 
>   client/src/main/resources/oozie-coordinator-0.5.xsd 
> e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 
>   core/pom.xml b063dab79415447a86c1a33f5c3f5304e0dffca0 
>   core/src/main/java/org/apache/oozie/CoordinatorActionBean.java 
> 91bff4dca2ef2dece68ca7260724ba3e43b1a08e 
>   core/src/main/java/org/apache/oozie/ErrorCode.java 
> 6c1e3997c9a1cb0bb0de39d687a083af0a7b5f04 
>   
> core/src/main/java/org/apache/oozie/command/coord/CoordActionInputCheckXCommand.java
>  742d00dd47aab55392abc8fbc207f8100728b832 
>   
> core/src/main/java/org/apache/oozie/command/coord/CoordActionUpdatePushMissingDependency.java
>  4e1c5b3392358cb6b1e98e16e469310338f27fed 
>   core/src/main/java/org/apache/oozie/command/coord/CoordCommandUtils.java 
> 58ef483272264e0d391d4a1e6533dc1cab9940da 
>   
> core/src/main/java/org/apache/oozie/command/coord/CoordMaterializeTransitionXCommand.java
>  39e6ac15ce3a3ea7f2ed9178688537f7b1d7842d 
>   
> core/src/main/java/org/apache/oozie/command/coord/CoordPushDependencyCheckXCommand.java
>  b05344d89e8df0e11fe69c1aa725d19a18eb0a2b 
>   core/src/main/java/org/apache/oozie/coord/CoordELConstants.java 
> f010a817fc900821c0e429fc16e1d3902a98d8bb 
>   core/src/main/java/org/apache/oozie/coord/CoordELEvaluator.java 
> 8b2f4560ae66bbcd707a446382e647663ea67be1 
>   core/src/main/java/org/apache/oozie

Re: Review Request 38474: OOZIE-1976- Specifying coordinator input datasets in more logical ways

[Rohini Palaniswamy]

dInputDependencyCheckPhaseOne.java
 (lines 155 - 156)
<https://reviews.apache.org/r/38474/#comment172767>

log message should be moved into if 
(!pathExists(coordInputInstance.getInputDataInstance(), jobConf)) block



core/src/main/java/org/apache/oozie/coord/input/dependency/CoordInputDependencyCheckPhaseOne.java
 (lines 247 - 251)
<https://reviews.apache.org/r/38474/#comment172606>

Method seems redundant as it just does call another method. Can be just 
inlined.



core/src/main/java/org/apache/oozie/coord/input/dependency/CoordInputDependencyCheckPhaseOne.java
 (line 286)
<https://reviews.apache.org/r/38474/#comment172770>

Using true and false as special case values in the string makes the code 
hacky and was also very hard to follow the code and understand the logic.

Can you use a object instead of String and store the path in one variable 
and this state in another.



core/src/main/java/org/apache/oozie/coord/input/dependency/CoordInputDependencyCheckPhaseTwo.java
 (line 73)
<https://reviews.apache.org/r/38474/#comment172785>

Why do we need RESOLVED_PATH_WITH_DONE_FLAG instead of RESOLVED_PATH?



core/src/main/java/org/apache/oozie/coord/input/dependency/CoordInputDependencyCheckPhaseTwo.java
 (line 116)
<https://reviews.apache.org/r/38474/#comment172774>

Why do we need to do this? Where do old instances come from? Can you 
elaborate that on the comment.



core/src/main/java/org/apache/oozie/coord/input/dependency/CoordOldInputDependency.java
 (line 138)
<https://reviews.apache.org/r/38474/#comment172614>

dependency -> unresolvedDependencies



core/src/main/java/org/apache/oozie/coord/input/dependency/CoordOldInputDependency.java
 (line 140)
<https://reviews.apache.org/r/38474/#comment172621>

Should be CoordELFunctions.INSTANCE_SEPARATOR



core/src/main/java/org/apache/oozie/coord/input/dependency/CoordOldInputDependency.java
 (line 282)
<https://reviews.apache.org/r/38474/#comment172782>

Use the new constant added for is_resolved



core/src/main/java/org/apache/oozie/coord/input/dependency/CoordOldInputDependency.java
 (line 290)
<https://reviews.apache.org/r/38474/#comment172783>

Use the new constant added for resolved_path



core/src/main/java/org/apache/oozie/coord/input/dependency/CoordUnResolvedInputDependency.java
 (line 35)
<https://reviews.apache.org/r/38474/#comment172625>

private



core/src/main/java/org/apache/oozie/coord/input/dependency/CoordUnResolvedInputDependency.java
 (lines 53 - 54)
<https://reviews.apache.org/r/38474/#comment172627>

Plural naming for variable and getter. Either dependencies or dependencyList



core/src/main/java/org/apache/oozie/coord/input/dependency/CoordUnResolvedInputDependency.java
 (line 71)
<https://reviews.apache.org/r/38474/#comment172628>

Avoid adding separator here.



core/src/main/java/org/apache/oozie/coord/input/dependency/OozieJexlInterpreter.java
 (line 132)
<https://reviews.apache.org/r/38474/#comment172638>

PHASE_TWO_EVALUATION and TIMED_WAITING could be part of a single enum 
CoordInputDependencyCheck.STATE

Can use static import of the constant to avoid repeating the class name 
everywhere.


- Rohini Palaniswamy


On Dec. 15, 2015, 7:33 p.m., Purshotam Shah wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/38474/
> ---
> 
> (Updated Dec. 15, 2015, 7:33 p.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-1976
> https://issues.apache.org/jira/browse/OOZIE-1976
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> There are three components in this patch
> 
> 1. User interface
> A new tag is added to coordinator.xml
> ex.
> 
> 
>   
>   "
>   
>
>
>   
>   
>
>
>  ;
> 
> 
> 
> input-check will have nested and/or/combine operation. It can have min and 
> wait at operator or at date-in.
> If input-check tag is missing then it consider to be old approach where all 
> data dependency are needed.
> 
> 2. Processing
> input-check is converted into logical expression
>   (a&&B)||(c&&d)
> We use jexl to parse the logical expression.
> 
> There are three phase in parsing.
> phase 1 : only resolved dataset are parsed ( only current).   
> phase 2 : once all current are resolved, then future/latest are parsed.
> phase 3 : Doesn't do any filecheck, just return what is being parsed by 
> phase1 and phase

Re: Review Request 38474: OOZIE-1976- Specifying coordinator input datasets in more logical ways

[Rohini Palaniswamy]

> On Dec. 29, 2015, 10:05 p.m., Rohini Palaniswamy wrote:
> > client/src/main/resources/oozie-coordinator-0.5.xsd, lines 116-117
> > <https://reviews.apache.org/r/38474/diff/3/?file=1165028#file1165028line116>
> >
> > min and wait seems to be supported only at the data-in level.

Dropped the issue. It is supported on and and or as well.


- Rohini


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38474/#review112102
---


On Dec. 15, 2015, 7:33 p.m., Purshotam Shah wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/38474/
> ---
> 
> (Updated Dec. 15, 2015, 7:33 p.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-1976
> https://issues.apache.org/jira/browse/OOZIE-1976
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> There are three components in this patch
> 
> 1. User interface
> A new tag is added to coordinator.xml
> ex.
> 
> 
>   
>   "
>   
>
>
>   
>   
>
>
>  ;
> 
> 
> 
> input-check will have nested and/or/combine operation. It can have min and 
> wait at operator or at date-in.
> If input-check tag is missing then it consider to be old approach where all 
> data dependency are needed.
> 
> 2. Processing
> input-check is converted into logical expression
>   (a&&B)||(c&&d)
> We use jexl to parse the logical expression.
> 
> There are three phase in parsing.
> phase 1 : only resolved dataset are parsed ( only current).   
> phase 2 : once all current are resolved, then future/latest are parsed.
> phase 3 : Doesn't do any filecheck, just return what is being parsed by 
> phase1 and phase2. Is used for EL functions
> 
> 
> 3. Storage.
> if inputcheck is enable, push_missing_dependencies and missing_dependencies 
> are serialized and stored in DB.
> If then not then it's old approach, where they are stored in plan text. This 
> is backward compatible.
> 
> 
> Diffs
> -
> 
>   client/src/main/resources/oozie-coordinator-0.5.xsd 
> e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 
>   core/pom.xml b063dab79415447a86c1a33f5c3f5304e0dffca0 
>   core/src/main/java/org/apache/oozie/CoordinatorActionBean.java 
> 91bff4dca2ef2dece68ca7260724ba3e43b1a08e 
>   core/src/main/java/org/apache/oozie/ErrorCode.java 
> 6c1e3997c9a1cb0bb0de39d687a083af0a7b5f04 
>   
> core/src/main/java/org/apache/oozie/command/coord/CoordActionInputCheckXCommand.java
>  742d00dd47aab55392abc8fbc207f8100728b832 
>   
> core/src/main/java/org/apache/oozie/command/coord/CoordActionUpdatePushMissingDependency.java
>  4e1c5b3392358cb6b1e98e16e469310338f27fed 
>   core/src/main/java/org/apache/oozie/command/coord/CoordCommandUtils.java 
> 58ef483272264e0d391d4a1e6533dc1cab9940da 
>   
> core/src/main/java/org/apache/oozie/command/coord/CoordMaterializeTransitionXCommand.java
>  39e6ac15ce3a3ea7f2ed9178688537f7b1d7842d 
>   
> core/src/main/java/org/apache/oozie/command/coord/CoordPushDependencyCheckXCommand.java
>  b05344d89e8df0e11fe69c1aa725d19a18eb0a2b 
>   core/src/main/java/org/apache/oozie/coord/CoordELConstants.java 
> f010a817fc900821c0e429fc16e1d3902a98d8bb 
>   core/src/main/java/org/apache/oozie/coord/CoordELEvaluator.java 
> 8b2f4560ae66bbcd707a446382e647663ea67be1 
>   core/src/main/java/org/apache/oozie/coord/CoordELFunctions.java 
> 5d238663aa94f0dd55a9190b60bfe621439c7b53 
>   core/src/main/java/org/apache/oozie/coord/CoordUtils.java 
> 94c69740618110ea180b188ab0c5a02db76f8b4d 
>   core/src/main/java/org/apache/oozie/coord/SyncCoordAction.java 
> 44258eb5be40bf6769e32c8780117e8533d80d7e 
>   
> core/src/main/java/org/apache/oozie/coord/input/dependency/AbstractCoordInputDependency.java
>  e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 
>   
> core/src/main/java/org/apache/oozie/coord/input/dependency/CoordInputDependenciesBuilder.java
>  e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 
>   
> core/src/main/java/org/apache/oozie/coord/input/dependency/CoordInputDependenciesChecker.java
>  e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 
>   
> core/src/main/java/org/apache/oozie/coord/input/dependency/CoordInputDependenciesUtil.java
>  e69de29bb2d1d6434b8b29ae7

Re: Review Request 38474: OOZIE-1976- Specifying coordinator input datasets in more logical ways

[Rohini Palaniswamy]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38474/#review112330
---



core/src/test/java/org/apache/oozie/command/coord/TestCoordCommandUtils.java 
<https://reviews.apache.org/r/38474/#comment172807>

Why remove the hourly part of the testcase?



core/src/test/java/org/apache/oozie/command/coord/TestCoordSubmitXCommand.java 
<https://reviews.apache.org/r/38474/#comment172794>

Why is this testcase removed?



core/src/test/java/org/apache/oozie/coord/input/dependency/TestCoordInputCheckPush.java
 (line 183)
<https://reviews.apache.org/r/38474/#comment172796>

Can we assert for actual value returned so that can also be verified? It 
will also be a test for the order in which datasets are returned. Same for  all 
the new input check testcases added.



core/src/test/java/org/apache/oozie/coord/input/dependency/TestCoordInputCheckPush.java
 (line 228)
<https://reviews.apache.org/r/38474/#comment172799>

If nested combine is not supported please add a TODO comment here



core/src/test/java/org/apache/oozie/coord/input/dependency/TestCoordinatorInputCheck.java
 (line 831)
<https://reviews.apache.org/r/38474/#comment172805>

Add TODO saying not supported now



core/src/test/resources/coord-action-sla.xml (line 26)
<https://reviews.apache.org/r/38474/#comment172806>

Good fixing this. But tow was this working before when # was the separator?


- Rohini Palaniswamy


On Dec. 15, 2015, 7:33 p.m., Purshotam Shah wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/38474/
> ---
> 
> (Updated Dec. 15, 2015, 7:33 p.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-1976
> https://issues.apache.org/jira/browse/OOZIE-1976
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> There are three components in this patch
> 
> 1. User interface
> A new tag is added to coordinator.xml
> ex.
> 
> 
>   
>   "
>   
>
>
>   
>   
>
>
>  ;
> 
> 
> 
> input-check will have nested and/or/combine operation. It can have min and 
> wait at operator or at date-in.
> If input-check tag is missing then it consider to be old approach where all 
> data dependency are needed.
> 
> 2. Processing
> input-check is converted into logical expression
>   (a&&B)||(c&&d)
> We use jexl to parse the logical expression.
> 
> There are three phase in parsing.
> phase 1 : only resolved dataset are parsed ( only current).   
> phase 2 : once all current are resolved, then future/latest are parsed.
> phase 3 : Doesn't do any filecheck, just return what is being parsed by 
> phase1 and phase2. Is used for EL functions
> 
> 
> 3. Storage.
> if inputcheck is enable, push_missing_dependencies and missing_dependencies 
> are serialized and stored in DB.
> If then not then it's old approach, where they are stored in plan text. This 
> is backward compatible.
> 
> 
> Diffs
> -
> 
>   client/src/main/resources/oozie-coordinator-0.5.xsd 
> e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 
>   core/pom.xml b063dab79415447a86c1a33f5c3f5304e0dffca0 
>   core/src/main/java/org/apache/oozie/CoordinatorActionBean.java 
> 91bff4dca2ef2dece68ca7260724ba3e43b1a08e 
>   core/src/main/java/org/apache/oozie/ErrorCode.java 
> 6c1e3997c9a1cb0bb0de39d687a083af0a7b5f04 
>   
> core/src/main/java/org/apache/oozie/command/coord/CoordActionInputCheckXCommand.java
>  742d00dd47aab55392abc8fbc207f8100728b832 
>   
> core/src/main/java/org/apache/oozie/command/coord/CoordActionUpdatePushMissingDependency.java
>  4e1c5b3392358cb6b1e98e16e469310338f27fed 
>   core/src/main/java/org/apache/oozie/command/coord/CoordCommandUtils.java 
> 58ef483272264e0d391d4a1e6533dc1cab9940da 
>   
> core/src/main/java/org/apache/oozie/command/coord/CoordMaterializeTransitionXCommand.java
>  39e6ac15ce3a3ea7f2ed9178688537f7b1d7842d 
>   
> core/src/main/java/org/apache/oozie/command/coord/CoordPushDependencyCheckXCommand.java
>  b05344d89e8df0e11fe69c1aa725d19a18eb0a2b 
>   core/src/main/java/org/apache/oozie/coord/CoordELConstants.java 
> f010a817fc900821c0e429fc16e1d3902a98d8bb 
>   core/src/main/java/org/apache/oozie/coord/CoordELEvaluator.java 
> 8b2f4560ae66bbcd707a446382e647663ea67be1

Re: Review Request 42335: OOZIE-2439: FS Action no longer uses name-node from global section or default NN

[Rohini Palaniswamy]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/42335/#review114642
---

Ship it!


Ship It!

- Rohini Palaniswamy


On Jan. 15, 2016, 2:17 a.m., Robert Kanter wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/42335/
> ---
> 
> (Updated Jan. 15, 2016, 2:17 a.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-2439
> https://issues.apache.org/jira/browse/OOZIE-2439
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> https://issues.apache.org/jira/browse/OOZIE-2439
> 
> 
> Diffs
> -
> 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsActionExecutor.java 
> b8649ac 
>   
> core/src/main/java/org/apache/oozie/workflow/lite/LiteWorkflowAppParser.java 
> 03c84f1 
>   
> core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java
>  1fc1736 
>   core/src/test/resources/wf-schema-fs-no-namenode-global.xml PRE-CREATION 
>   core/src/test/resources/wf-schema-fs-no-namenode.xml PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/42335/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Robert Kanter
> 
>