Re: Review Request 35488: Fixed spark action in yarn mode and standalone mode

[Shwetha GS]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/35488/#review91456
---

Ship it!


Ship It!

- Shwetha GS


On June 24, 2015, 7:01 a.m., pavan kumar kolamuri wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/35488/
> ---
> 
> (Updated June 24, 2015, 7:01 a.m.)
> 
> 
> Review request for oozie and Srikanth Sundarrajan.
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> Spark job launched through spark action is not running as expected in 
> standalone mode and  it is not running in yarn mode
> 
> 
> Diffs
> -
> 
>   pom.xml ee26366 
>   sharelib/spark/pom.xml 3fe9c71 
>   src/main/assemblies/sharelib.xml ea95c2e 
> 
> Diff: https://reviews.apache.org/r/35488/diff/
> 
> 
> Testing
> ---
> 
> Basic testing done. Deployed spark job on yarn, standalone cluster and local 
> mode . It is running successfully in all these modes.
> 
> 
> Thanks,
> 
> pavan kumar kolamuri
> 
>

Re: Review Request 35379: OOZIE-2030 Configuration properties from global section is not getting set in Hadoop job conf when using sub-workflow action in Oozie workflow.xml

[Shwetha GS]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/35379/#review90683
---

Ship it!


Ship It!

- Shwetha GS


On July 7, 2015, 8:57 a.m., Jaydeep Vishwakarma wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/35379/
> ---
> 
> (Updated July 7, 2015, 8:57 a.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-2030
> https://issues.apache.org/jira/browse/OOZIE-2030
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> Forwarding global conf from workflow to action when action is subworkflow, 
> subworkflow storing it to property and forwarding as property. 
>  LiteworkflowAppParser handling all condition and ensuring child level pass 
> of global conf.
> 
> 
> Diffs
> -
> 
>   
> core/src/main/java/org/apache/oozie/workflow/lite/LiteWorkflowAppParser.java 
> c857011 
>   
> core/src/test/java/org/apache/oozie/action/oozie/TestSubWorkflowActionExecutor.java
>  9ab897a 
> 
> Diff: https://reviews.apache.org/r/35379/diff/
> 
> 
> Testing
> ---
> 
> yes
> 
> 
> Thanks,
> 
> Jaydeep Vishwakarma
> 
>

Re: Review Request 35379: OOZIE-2030 Configuration properties from global section is not getting set in Hadoop job conf when using sub-workflow action in Oozie workflow.xml

[Shwetha GS]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/35379/#review90671
---



core/src/main/java/org/apache/oozie/workflow/lite/LiteWorkflowAppParser.java 
(line 95)
<https://reviews.apache.org/r/35379/#comment143804>

rename to oozie.globalconf



core/src/main/java/org/apache/oozie/workflow/lite/LiteWorkflowAppParser.java 
(line 488)
<https://reviews.apache.org/r/35379/#comment143806>

add a comment that this merges the parent global conf and current global 
conf. Then sets the merged global conf in jobConf


Minor comments, looks good otherwise. Simpler and nice approach. Thanks

- Shwetha GS


On June 16, 2015, 12:06 p.m., Jaydeep Vishwakarma wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/35379/
> ---
> 
> (Updated June 16, 2015, 12:06 p.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-2030
> https://issues.apache.org/jira/browse/OOZIE-2030
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> Forwarding global conf from workflow to action when action is subworkflow, 
> subworkflow storing it to property and forwarding as property. 
>  LiteworkflowAppParser handling all condition and ensuring child level pass 
> of global conf.
> 
> 
> Diffs
> -
> 
>   
> core/src/main/java/org/apache/oozie/workflow/lite/LiteWorkflowAppParser.java 
> c857011 
>   
> core/src/test/java/org/apache/oozie/action/oozie/TestSubWorkflowActionExecutor.java
>  9ab897a 
> 
> Diff: https://reviews.apache.org/r/35379/diff/
> 
> 
> Testing
> ---
> 
> yes
> 
> 
> Thanks,
> 
> Jaydeep Vishwakarma
> 
>

Re: Review Request 35488: Fixed spark action in yarn mode and standalone mode

[Shwetha GS]

> On June 23, 2015, 7:09 a.m., pavan kumar kolamuri wrote:
> > sharelib/spark/pom.xml, line 111
> > 
> >
> > Yes i agree with that, it is same across all other actions as well. 
> > Refactoring is required for other actions as well , we can raise a jira and 
> > fix all this dependency management issues

Atleast the versions need to move to dependency management - just like other 
actions


- Shwetha


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/35488/#review88927
---


On June 16, 2015, 5:09 a.m., pavan kumar kolamuri wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/35488/
> ---
> 
> (Updated June 16, 2015, 5:09 a.m.)
> 
> 
> Review request for oozie and Srikanth Sundarrajan.
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> Spark job launched through spark action is not running as expected in 
> standalone mode and  it is not running in yarn mode
> 
> 
> Diffs
> -
> 
>   pom.xml ee26366 
>   sharelib/spark/pom.xml 3fe9c71 
>   src/main/assemblies/sharelib.xml ea95c2e 
> 
> Diff: https://reviews.apache.org/r/35488/diff/
> 
> 
> Testing
> ---
> 
> Basic testing done. Deployed spark job on yarn, standalone cluster and local 
> mode . It is running successfully in all these modes.
> 
> 
> Thanks,
> 
> pavan kumar kolamuri
> 
>

Re: Review Request 35488: Fixed spark action in yarn mode and standalone mode

[Shwetha GS]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/35488/#review88923
---



sharelib/spark/pom.xml (line 107)
<https://reviews.apache.org/r/35488/#comment141539>

can you move the exclusions to parent pom?



sharelib/spark/pom.xml (line 224)
<https://reviews.apache.org/r/35488/#comment141540>

why are you creating uber jar? uber jars are difficult to debug - can't 
figure out what jars and classes are added


- Shwetha GS


On June 16, 2015, 5:09 a.m., pavan kumar kolamuri wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/35488/
> ---
> 
> (Updated June 16, 2015, 5:09 a.m.)
> 
> 
> Review request for oozie and Srikanth Sundarrajan.
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> Spark job launched through spark action is not running as expected in 
> standalone mode and  it is not running in yarn mode
> 
> 
> Diffs
> -
> 
>   pom.xml ee26366 
>   sharelib/spark/pom.xml 3fe9c71 
>   src/main/assemblies/sharelib.xml ea95c2e 
> 
> Diff: https://reviews.apache.org/r/35488/diff/
> 
> 
> Testing
> ---
> 
> Basic testing done. Deployed spark job on yarn, standalone cluster and local 
> mode . It is running successfully in all these modes.
> 
> 
> Thanks,
> 
> pavan kumar kolamuri
> 
>

Re: Review Request 34854: OOZIE-2251 : Expose instrumental matrices in Realtime Graphing tool

[Shwetha GS]

> On June 15, 2015, 6:25 a.m., Shwetha GS wrote:
> > core/src/main/java/org/apache/oozie/util/MetricsInstrumentation.java, line 
> > 98
> > <https://reviews.apache.org/r/34854/diff/7/?file=983898#file983898line98>
> >
> > prefix all related configs with 'oozie.external_monitoring.'. For 
> > example, change oozie.metrics.report.interval.sec - 
> > oozie.external_monitoring.reporterIntervalSecs
> > oozie.metrics.host - oozie.external_monitoring.address (covers both 
> > host and port so that there are less configs that users need to set)
> > oozie.metrics.server.name - oozie.external_monitoring.type
> > oozie.metrics.prefix - oozie.external_monitoring.metricPrefix
> > 
> > Use . for hierarchical configs, not as word separator
> 
> Narayan Periwal wrote:
> When we initialize the graphite object, there we need to pass the host 
> and the port separately to make the InetSocketAddress object. So, to avoid 
> the parsing, I am keeping the host and the port as separate configs. Also, 
> now the user cannot fail to give the port number.
> 
> Narayan Periwal wrote:
> Please let me know if you have some other suggestion

For the function, you can use URL and get host and port. We should minimise the 
number of configs that users have to set. 

now the user cannot fail to give the port number - why can't they skip setting 
another config?

Both graphite and ganglia servers have different default ports than the default 
that you have set.


- Shwetha


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34854/#review87894
---


On June 12, 2015, 7:53 p.m., Narayan Periwal wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/34854/
> ---
> 
> (Updated June 12, 2015, 7:53 p.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-2251
> https://issues.apache.org/jira/browse/OOZIE-2251
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> We have been logging so many important matrices in oozie-instrumentation.log 
> . These information is very useful for oozie functional monitoring. But it is 
> always difficult to get the meaning from flat file. If we expose this 
> information on some graphing tool, We can get the lot of meaning out of it 
> and can take some actions based on it.
> 
> 
> Diffs
> -
> 
>   core/pom.xml 783 
>   
> core/src/main/java/org/apache/oozie/service/MetricsInstrumentationService.java
>  29f6294 
>   core/src/main/java/org/apache/oozie/util/Instrumentation.java 3dfb67a 
>   core/src/main/java/org/apache/oozie/util/MetricsInstrumentation.java 
> e56bfda 
>   core/src/main/resources/oozie-default.xml 8960073 
>   docs/src/site/twiki/AG_Install.twiki 0ce2609 
> 
> Diff: https://reviews.apache.org/r/34854/diff/
> 
> 
> Testing
> ---
> 
> Done
> 
> 
> Thanks,
> 
> Narayan Periwal
> 
>

Re: Review Request 34854: OOZIE-2251 : Expose instrumental matrices in Realtime Graphing tool

[Shwetha GS]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34854/#review87894
---


Some minor comments. In general, avoid extra code so that code is more readable 
and maintainable


core/src/main/java/org/apache/oozie/util/MetricsInstrumentation.java
<https://reviews.apache.org/r/34854/#comment140301>

'== true' is not required



core/src/main/java/org/apache/oozie/util/MetricsInstrumentation.java
<https://reviews.apache.org/r/34854/#comment140302>

prefix all related configs with 'oozie.external_monitoring.'. For example, 
change oozie.metrics.report.interval.sec - 
oozie.external_monitoring.reporterIntervalSecs
oozie.metrics.host - oozie.external_monitoring.address (covers both host 
and port so that there are less configs that users need to set)
oozie.metrics.server.name - oozie.external_monitoring.type
oozie.metrics.prefix - oozie.external_monitoring.metricPrefix

Use . for hierarchical configs, not as word separator



core/src/main/java/org/apache/oozie/util/MetricsInstrumentation.java
<https://reviews.apache.org/r/34854/#comment140306>

default is STD, skip settting it



core/src/main/java/org/apache/oozie/util/MetricsInstrumentation.java
<https://reviews.apache.org/r/34854/#comment140303>

change log level to debug



core/src/main/java/org/apache/oozie/util/MetricsInstrumentation.java
<https://reviews.apache.org/r/34854/#comment140304>

Combine reading confs and reporter initialisation to one if block



core/src/main/java/org/apache/oozie/util/MetricsInstrumentation.java
<https://reviews.apache.org/r/34854/#comment140307>

use LOG



core/src/main/java/org/apache/oozie/util/MetricsInstrumentation.java
<https://reviews.apache.org/r/34854/#comment140308>

Move the common code of the if blocks to outside if block. You can create 
the graphite/ganlia registry inside if blocks and rest outside



core/src/main/resources/oozie-default.xml
<https://reviews.apache.org/r/34854/#comment140305>

default to just 'oozie'?


- Shwetha GS


On June 12, 2015, 7:53 p.m., Narayan Periwal wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/34854/
> ---
> 
> (Updated June 12, 2015, 7:53 p.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-2251
> https://issues.apache.org/jira/browse/OOZIE-2251
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> We have been logging so many important matrices in oozie-instrumentation.log 
> . These information is very useful for oozie functional monitoring. But it is 
> always difficult to get the meaning from flat file. If we expose this 
> information on some graphing tool, We can get the lot of meaning out of it 
> and can take some actions based on it.
> 
> 
> Diffs
> -
> 
>   core/pom.xml 783 
>   
> core/src/main/java/org/apache/oozie/service/MetricsInstrumentationService.java
>  29f6294 
>   core/src/main/java/org/apache/oozie/util/Instrumentation.java 3dfb67a 
>   core/src/main/java/org/apache/oozie/util/MetricsInstrumentation.java 
> e56bfda 
>   core/src/main/resources/oozie-default.xml 8960073 
>   docs/src/site/twiki/AG_Install.twiki 0ce2609 
> 
> Diff: https://reviews.apache.org/r/34854/diff/
> 
> 
> Testing
> ---
> 
> Done
> 
> 
> Thanks,
> 
> Narayan Periwal
> 
>

Re: Review Request 33624: OOZIE-1993 Rerun fails during join in certain condition

[Shwetha GS]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/33624/
---

(Updated May 18, 2015, 8:59 a.m.)


Review request for oozie.


Changes
---

Added test


Bugs: OOZIE-1993
https://issues.apache.org/jira/browse/OOZIE-1993


Repository: oozie-git


Description
---

The issue was with picking up new execution path during re-runs. This will also 
address the issue in OOZIE-1879.
Changes:
1. SignalXCommand to pick up the new execution path
2. Removed actionEndTimes comparison
3. Aligned else ifs in LiteWorkflowInstance.signal(). No logic changes


Diffs (updated)
-

  core/src/main/java/org/apache/oozie/WorkflowActionBean.java 6289158 
  core/src/main/java/org/apache/oozie/command/wf/ReRunXCommand.java 6138278 
  core/src/main/java/org/apache/oozie/command/wf/SignalXCommand.java bccac51 
  
core/src/main/java/org/apache/oozie/executor/jpa/WorkflowActionQueryExecutor.java
 0e99ae2 
  core/src/main/java/org/apache/oozie/service/LiteWorkflowStoreService.java 
d661d08 
  core/src/main/java/org/apache/oozie/workflow/WorkflowLib.java ba5b6b1 
  core/src/main/java/org/apache/oozie/workflow/lite/LiteWorkflowInstance.java 
919c95a 
  core/src/main/java/org/apache/oozie/workflow/lite/LiteWorkflowLib.java 
706cc2a 
  core/src/main/resources/oozie-log4j.properties 4b532ca 
  core/src/test/java/org/apache/oozie/command/wf/TestReRunXCommand.java ce32e51 
  core/src/test/resources/rerun-wf-fork.xml 8fa8f34 

Diff: https://reviews.apache.org/r/33624/diff/


Testing
---

end to end test for the example workflow in the jira - with failures at 
different nodes


Thanks,

Shwetha GS

Re: Review Request 33624: OOZIE-1993 Rerun fails during join in certain condition

[Shwetha GS]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/33624/
---

(Updated April 30, 2015, 7:14 a.m.)


Review request for oozie.


Changes
---

Removed log4j file changes


Bugs: OOZIE-1993
https://issues.apache.org/jira/browse/OOZIE-1993


Repository: oozie-git


Description
---

The issue was with picking up new execution path during re-runs. This will also 
address the issue in OOZIE-1879.
Changes:
1. SignalXCommand to pick up the new execution path
2. Removed actionEndTimes comparison
3. Aligned else ifs in LiteWorkflowInstance.signal(). No logic changes


Diffs (updated)
-

  core/src/main/java/org/apache/oozie/WorkflowActionBean.java 6289158 
  core/src/main/java/org/apache/oozie/command/wf/ReRunXCommand.java 6138278 
  core/src/main/java/org/apache/oozie/command/wf/SignalXCommand.java bccac51 
  
core/src/main/java/org/apache/oozie/executor/jpa/WorkflowActionQueryExecutor.java
 0e99ae2 
  core/src/main/java/org/apache/oozie/service/LiteWorkflowStoreService.java 
d661d08 
  core/src/main/java/org/apache/oozie/workflow/WorkflowLib.java ba5b6b1 
  core/src/main/java/org/apache/oozie/workflow/lite/LiteWorkflowInstance.java 
919c95a 
  core/src/main/java/org/apache/oozie/workflow/lite/LiteWorkflowLib.java 
706cc2a 

Diff: https://reviews.apache.org/r/33624/diff/


Testing
---

end to end test for the example workflow in the jira - with failures at 
different nodes


Thanks,

Shwetha GS

Re: Review Request 33624: OOZIE-1993 Rerun fails during join in certain condition

[Shwetha GS]

> On April 28, 2015, 9:13 p.m., Robert Kanter wrote:
> >

Does the main code change look good?


> On April 28, 2015, 9:13 p.m., Robert Kanter wrote:
> > core/src/main/conf/oozie-log4j.properties, line 40
> > <https://reviews.apache.org/r/33624/diff/1/?file=943971#file943971line40>
> >
> > Why are we changing the log4j config here?

Not required for this patch. I had plans of doing this in another jira which I 
never did. Added this during testing, so came in the patch. If you have 
concerns about adding in this patch, I can revert


- Shwetha


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/33624/#review81883
-------


On April 28, 2015, 11:16 a.m., Shwetha GS wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/33624/
> ---
> 
> (Updated April 28, 2015, 11:16 a.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-1993
> https://issues.apache.org/jira/browse/OOZIE-1993
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> The issue was with picking up new execution path during re-runs. This will 
> also address the issue in OOZIE-1879.
> Changes:
> 1. SignalXCommand to pick up the new execution path
> 2. Removed actionEndTimes comparison
> 3. Aligned else ifs in LiteWorkflowInstance.signal(). No logic changes
> 
> 
> Diffs
> -
> 
>   core/src/main/conf/oozie-log4j.properties 2b20ff2 
>   core/src/main/java/org/apache/oozie/WorkflowActionBean.java 6289158 
>   core/src/main/java/org/apache/oozie/command/wf/ReRunXCommand.java 6138278 
>   core/src/main/java/org/apache/oozie/command/wf/SignalXCommand.java bccac51 
>   
> core/src/main/java/org/apache/oozie/executor/jpa/WorkflowActionQueryExecutor.java
>  0e99ae2 
>   core/src/main/java/org/apache/oozie/service/LiteWorkflowStoreService.java 
> d661d08 
>   core/src/main/java/org/apache/oozie/workflow/WorkflowLib.java ba5b6b1 
>   core/src/main/java/org/apache/oozie/workflow/lite/LiteWorkflowInstance.java 
> 919c95a 
>   core/src/main/java/org/apache/oozie/workflow/lite/LiteWorkflowLib.java 
> 706cc2a 
> 
> Diff: https://reviews.apache.org/r/33624/diff/
> 
> 
> Testing
> ---
> 
> end to end test for the example workflow in the jira - with failures at 
> different nodes
> 
> 
> Thanks,
> 
> Shwetha GS
> 
>

Review Request 33624: OOZIE-1993 Rerun fails during join in certain condition

[Shwetha GS]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/33624/
---

Review request for oozie.


Bugs: OOZIE-1993
https://issues.apache.org/jira/browse/OOZIE-1993


Repository: oozie-git


Description
---

The issue was with picking up new execution path during re-runs. This will also 
address the issue in OOZIE-1879.
Changes:
1. SignalXCommand to pick up the new execution path
2. Removed actionEndTimes comparison
3. Aligned else ifs in LiteWorkflowInstance.signal(). No logic changes


Diffs
-

  core/src/main/conf/oozie-log4j.properties 2b20ff2 
  core/src/main/java/org/apache/oozie/WorkflowActionBean.java 6289158 
  core/src/main/java/org/apache/oozie/command/wf/ReRunXCommand.java 6138278 
  core/src/main/java/org/apache/oozie/command/wf/SignalXCommand.java bccac51 
  
core/src/main/java/org/apache/oozie/executor/jpa/WorkflowActionQueryExecutor.java
 0e99ae2 
  core/src/main/java/org/apache/oozie/service/LiteWorkflowStoreService.java 
d661d08 
  core/src/main/java/org/apache/oozie/workflow/WorkflowLib.java ba5b6b1 
  core/src/main/java/org/apache/oozie/workflow/lite/LiteWorkflowInstance.java 
919c95a 
  core/src/main/java/org/apache/oozie/workflow/lite/LiteWorkflowLib.java 
706cc2a 

Diff: https://reviews.apache.org/r/33624/diff/


Testing
---

end to end test for the example workflow in the jira - with failures at 
different nodes


Thanks,

Shwetha GS

Re: Review Request 28637: OOZIE-2030 Configuration properties from global section is not getting set in Hadoop job conf when using sub-workflow action in Oozie workflow.xml

[shwetha . gs]

> On Dec. 11, 2014, 9:21 p.m., Rohini Palaniswamy wrote:
> > core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java, 
> > line 242
> > 
> >
> > I think it actually makes sense and whatever I thought might conflict 
> > would be ok. Could you still add a config 
> > oozie.launcher.only.job.copy.actionconf (or some better name) as bother 
> > server and action level config with default value as true, so that we can 
> > turn it off in production if some unexpected conflict rises. If only few 
> > users encounter issue we can ask them to override at action level and if 
> > there is a major issue can turn it off at server level itself.

Will do


> On Dec. 11, 2014, 9:21 p.m., Rohini Palaniswamy wrote:
> > core/src/main/java/org/apache/oozie/action/oozie/SubWorkflowActionExecutor.java,
> >  line 180
> > 
> >
> > Based on the code in LiteWorkflowAppParser.handleGlobal, 
> > eConf.getChild("configuration", ns) should contain the global 
> > configuration. Instead of parsing global and jobxml again, you could just 
> > use that.
> > 
> >   I think the cleaner way to do this would be to not touch 
> > ActionExecutors at all. Should parse and store global configuration in 
> > WorkflowInstance once and add it to each action configuration and also 
> > initialize global configuration of sub WorkflowInstance created with it. 
> > Currently the global section and its job.xml is parsed for each action 
> > which is really bad for latency. This would also take care of propagating 
> > it to nested sub-workflows. If the sub-workflow had global configuration 
> > then that should be merged while parsing itself to the parent global config 
> > initialized.

Makes sense. But that change will be more involved. Lets track that in another 
jira


- shwethags


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28637/#review64788
---


On Dec. 8, 2014, 7:20 a.m., shwethags wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28637/
> ---
> 
> (Updated Dec. 8, 2014, 7:20 a.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-2030
> https://issues.apache.org/jira/browse/OOZIE-2030
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> 1. Propagate properties from configuration and job-xml from parent workflow 
> to sub-workflow. Currently, its propagated as workflow properties, but they 
> are required in action conf
> 2. For java actions, add properties from configuration and job-xml to 
> launcher job
> 
> 
> Diffs
> -
> 
>   core/src/main/java/org/apache/oozie/action/ActionExecutor.java ff836fb 
>   core/src/main/java/org/apache/oozie/action/hadoop/FsActionExecutor.java 
> fed1d7a 
>   core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 
> f207d74 
>   
> core/src/main/java/org/apache/oozie/action/oozie/SubWorkflowActionExecutor.java
>  bda34b5 
>   core/src/main/java/org/apache/oozie/service/HadoopAccessorService.java 
> 18de48a 
>   
> core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java 
> 48166a5 
>   
> core/src/test/java/org/apache/oozie/action/oozie/TestSubWorkflowActionExecutor.java
>  0d7e926 
>   examples/src/main/apps/subwf/job.xml PRE-CREATION 
>   examples/src/main/apps/subwf/workflow.xml 9873a50 
> 
> Diff: https://reviews.apache.org/r/28637/diff/
> 
> 
> Testing
> ---
> 
> UTs and end to end test
> 
> 
> Thanks,
> 
> shwethags
> 
>

Re: Review Request 22615: OOZIE-1876 use pom properties rather than specific version numbers in the pom files of hbaselibs, hcataloglibs, sharelib, etc

[shwetha . gs]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/22615/
---

(Updated Dec. 9, 2014, 12:01 p.m.)


Review request for oozie.


Changes
---

Added profile for bundling hadoop & hcat libraries


Bugs: OOZIE-1876
https://issues.apache.org/jira/browse/OOZIE-1876


Repository: oozie-git


Description
---

1. hbase/hcatalog versions configurable
2. Include hcatalog libs in final distro


Diffs (updated)
-

  client/src/main/bin/oozie 44943fa 
  core/pom.xml ca40e2e 
  core/src/main/conf/hadoop-conf/core-site.xml 35078c7 
  core/src/main/conf/hadoop-conf/core-site.xml.template PRE-CREATION 
  core/src/test/java/org/apache/oozie/test/XTestCase.java 27cbdc9 
  distro/src/main/bin/oozie-setup.sh 45f21de 
  distro/src/main/bin/oozied.sh 8eafc96 
  docs/src/site/twiki/DG_QuickStart.twiki 526fbc3 
  docs/src/site/twiki/ENG_Building.twiki 8d74b72 
  examples/pom.xml 852d222 
  hadooplibs/hadoop-0.23/pom.xml 6297dbd 
  hadooplibs/hadoop-1/pom.xml f0a32a5 
  hadooplibs/hadoop-2/pom.xml 32d4b6c 
  hadooplibs/hadoop-3/pom.xml 4fd8f29 
  hadooplibs/hadoop-auth-0.23/pom.xml 59cc7f1 
  hadooplibs/hadoop-auth-1/pom.xml 7da7639 
  hadooplibs/hadoop-auth-2/pom.xml cbdb523 
  hadooplibs/hadoop-distcp-0.23/pom.xml 6e42986 
  hadooplibs/hadoop-distcp-1/pom.xml 01eb123 
  hadooplibs/hadoop-distcp-2/pom.xml 7971eb5 
  hadooplibs/hadoop-distcp-3/pom.xml 598ee13 
  hadooplibs/hadoop-test-0.23/pom.xml addcd11 
  hadooplibs/hadoop-test-1/pom.xml baf5a4c 
  hadooplibs/hadoop-test-2/pom.xml 027c813 
  hadooplibs/hadoop-test-3/pom.xml 7176f51 
  hadooplibs/hadoop-utils-0.23/pom.xml 91985fc 
  hadooplibs/hadoop-utils-1/pom.xml c81ea91 
  hadooplibs/hadoop-utils-2/pom.xml 2d3dbdd 
  hadooplibs/hadoop-utils-3/pom.xml 7270bfd 
  hadooplibs/pom.xml e5ea2e7 
  hbaselibs/hbase-0.94/pom.xml a65904e 
  hbaselibs/pom.xml 24051c5 
  hcataloglibs/hive-hcatalog-0.13.1/pom.xml b738741 
  hcataloglibs/pom.xml a96806b 
  minitest/pom.xml 8f656d2 
  pom.xml 5507381 
  sharelib/distcp/pom.xml b788ed0 
  sharelib/hcatalog/pom.xml b934be1 
  sharelib/hive/pom.xml ea088ff 
  sharelib/hive2/pom.xml e4c2465 
  sharelib/oozie/pom.xml 053cf1c 
  sharelib/pig/pom.xml ce75c68 
  sharelib/spark/pom.xml 744c95c 
  sharelib/sqoop/pom.xml 09f0b66 
  sharelib/streaming/pom.xml 50d15f7 
  src/main/assemblies/distro.xml 8e2bbae 
  src/main/assemblies/hadooplib.xml b32ca82 
  src/main/assemblies/hadooplibs.xml 9884efa 
  src/main/assemblies/hbaselib.xml 6e4db33 
  src/main/assemblies/hbaselibs.xml eb71624 
  src/main/assemblies/hcataloglib.xml 1002308 
  src/main/assemblies/hcataloglibs.xml 3d32a8f 
  tools/pom.xml 80fa425 
  tools/src/main/bin/ooziedb.sh 68158d8 
  tools/src/main/java/org/apache/oozie/tools/OozieDBCLI.java ca80d35 
  webapp/pom.xml a410b10 
  zookeeper-security-tests/pom.xml b24ce33 

Diff: https://reviews.apache.org/r/22615/diff/


Testing
---

Clean build with different hive and hbase versions


Thanks,

shwethags

OOZIE-1876 Oozie packaging changes

[Shwetha GS]

Hi All,

Currently, oozie war doesn't contain any hadoop and hcat client libraries
and they need to be added to libext at install time. Since users/admins
will not know the exact jars required, they end up adding hadoop and hcat
classpath which contains even server jars. At the end, oozie war will have
multiple versions of the same jars and creates issues. This is more evident
with hadoop-2 and hive-0.13.

With OOZIE-1876, oozie needs to be built with specific hadoop and hcat
versions required and the war bundles all the required jars. So, its not
possible to use the same oozie package for multiple hadoop versions.

Robert has reviewed it and the patch is ready for commit. If anyone has any
concerns with the approach, please let me know.

Thanks,
Shwetha

-- 
_
The information contained in this communication is intended solely for the 
use of the individual or entity to whom it is addressed and others 
authorized to receive it. It may contain confidential or legally privileged 
information. If you are not the intended recipient you are hereby notified 
that any disclosure, copying, distribution or taking any action in reliance 
on the contents of this information is strictly prohibited and may be 
unlawful. If you have received this communication in error, please notify 
us immediately by responding to this email and then delete it from your 
system. The firm is neither liable for the proper and complete transmission 
of the information contained in this communication nor for any delay in its 
receipt.

Re: Review Request 28637: OOZIE-2030 Configuration properties from global section is not getting set in Hadoop job conf when using sub-workflow action in Oozie workflow.xml

[shwetha . gs]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28637/
---

(Updated Dec. 8, 2014, 7:20 a.m.)


Review request for oozie.


Changes
---

Moved parseJobXmlAndConfiguration() from JavaActionExecutor to ActionExecutor


Bugs: OOZIE-2030
https://issues.apache.org/jira/browse/OOZIE-2030


Repository: oozie-git


Description
---

1. Propagate properties from configuration and job-xml from parent workflow to 
sub-workflow. Currently, its propagated as workflow properties, but they are 
required in action conf
2. For java actions, add properties from configuration and job-xml to launcher 
job


Diffs (updated)
-

  core/src/main/java/org/apache/oozie/action/ActionExecutor.java ff836fb 
  core/src/main/java/org/apache/oozie/action/hadoop/FsActionExecutor.java 
fed1d7a 
  core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 
f207d74 
  
core/src/main/java/org/apache/oozie/action/oozie/SubWorkflowActionExecutor.java 
bda34b5 
  core/src/main/java/org/apache/oozie/service/HadoopAccessorService.java 
18de48a 
  core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java 
48166a5 
  
core/src/test/java/org/apache/oozie/action/oozie/TestSubWorkflowActionExecutor.java
 0d7e926 
  examples/src/main/apps/subwf/job.xml PRE-CREATION 
  examples/src/main/apps/subwf/workflow.xml 9873a50 

Diff: https://reviews.apache.org/r/28637/diff/


Testing
---

UTs and end to end test


Thanks,

shwethags

Review Request 28637: OOZIE-2030 Configuration properties from global section is not getting set in Hadoop job conf when using sub-workflow action in Oozie workflow.xml

[shwetha . gs]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28637/
---

Review request for oozie.


Bugs: OOZIE-2030
https://issues.apache.org/jira/browse/OOZIE-2030


Repository: oozie-git


Description
---

1. Propagate properties from configuration and job-xml from parent workflow to 
sub-workflow. Currently, its propagated as workflow properties, but they are 
required in action conf
2. For java actions, add properties from configuration and job-xml to launcher 
job


Diffs
-

  core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 
f207d74 
  core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 
f207d74 
  
core/src/main/java/org/apache/oozie/action/oozie/SubWorkflowActionExecutor.java 
bda34b5 
  
core/src/main/java/org/apache/oozie/action/oozie/SubWorkflowActionExecutor.java 
bda34b5 
  core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java 
48166a5 
  core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java 
48166a5 
  
core/src/test/java/org/apache/oozie/action/oozie/TestSubWorkflowActionExecutor.java
 0d7e926 
  
core/src/test/java/org/apache/oozie/action/oozie/TestSubWorkflowActionExecutor.java
 0d7e926 
  examples/src/main/apps/subwf/job.xml PRE-CREATION 
  examples/src/main/apps/subwf/workflow.xml 9873a50 
  examples/src/main/apps/subwf/workflow.xml 9873a50 

Diff: https://reviews.apache.org/r/28637/diff/


Testing
---

UTs and end to end test


Thanks,

shwethags

Re: Review Request 28637: OOZIE-2030 Configuration properties from global section is not getting set in Hadoop job conf when using sub-workflow action in Oozie workflow.xml

[shwetha . gs]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28637/
---

(Updated Dec. 3, 2014, 9:20 a.m.)


Review request for oozie.


Bugs: OOZIE-2030
https://issues.apache.org/jira/browse/OOZIE-2030


Repository: oozie-git


Description
---

1. Propagate properties from configuration and job-xml from parent workflow to 
sub-workflow. Currently, its propagated as workflow properties, but they are 
required in action conf
2. For java actions, add properties from configuration and job-xml to launcher 
job


Diffs (updated)
-

  core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 
f207d74 
  
core/src/main/java/org/apache/oozie/action/oozie/SubWorkflowActionExecutor.java 
bda34b5 
  core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java 
48166a5 
  
core/src/test/java/org/apache/oozie/action/oozie/TestSubWorkflowActionExecutor.java
 0d7e926 
  examples/src/main/apps/subwf/job.xml PRE-CREATION 
  examples/src/main/apps/subwf/workflow.xml 9873a50 

Diff: https://reviews.apache.org/r/28637/diff/


Testing
---

UTs and end to end test


Thanks,

shwethags

Re: Review Request 22615: OOZIE-1876 use pom properties rather than specific version numbers in the pom files of hbaselibs, hcataloglibs, sharelib, etc

[shwetha . gs]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/22615/
---

(Updated Nov. 27, 2014, 1:04 p.m.)


Review request for oozie.


Changes
---

verified the sharelibs. All sharelibs look the same except for pig. For pig, I 
verified with a pig job and works fine.


Bugs: OOZIE-1876
https://issues.apache.org/jira/browse/OOZIE-1876


Repository: oozie-git


Description
---

1. hbase/hcatalog versions configurable
2. Include hcatalog libs in final distro


Diffs (updated)
-

  client/src/main/bin/oozie 44943fa 
  core/pom.xml ca40e2e 
  core/src/main/conf/hadoop-conf/core-site.xml 35078c7 
  core/src/main/conf/hadoop-conf/core-site.xml.template PRE-CREATION 
  core/src/test/java/org/apache/oozie/test/XTestCase.java 27cbdc9 
  distro/src/main/bin/oozie-setup.sh 45f21de 
  distro/src/main/bin/oozied.sh 8eafc96 
  docs/src/site/twiki/DG_QuickStart.twiki 526fbc3 
  docs/src/site/twiki/ENG_Building.twiki 8d74b72 
  examples/pom.xml 852d222 
  hadooplibs/hadoop-0.23/pom.xml 6297dbd 
  hadooplibs/hadoop-1/pom.xml f0a32a5 
  hadooplibs/hadoop-2/pom.xml 32d4b6c 
  hadooplibs/hadoop-3/pom.xml 4fd8f29 
  hadooplibs/hadoop-auth-0.23/pom.xml 59cc7f1 
  hadooplibs/hadoop-auth-1/pom.xml 7da7639 
  hadooplibs/hadoop-auth-2/pom.xml cbdb523 
  hadooplibs/hadoop-distcp-0.23/pom.xml 6e42986 
  hadooplibs/hadoop-distcp-1/pom.xml 01eb123 
  hadooplibs/hadoop-distcp-2/pom.xml 7971eb5 
  hadooplibs/hadoop-distcp-3/pom.xml 598ee13 
  hadooplibs/hadoop-test-0.23/pom.xml addcd11 
  hadooplibs/hadoop-test-1/pom.xml baf5a4c 
  hadooplibs/hadoop-test-2/pom.xml 027c813 
  hadooplibs/hadoop-test-3/pom.xml 7176f51 
  hadooplibs/hadoop-utils-0.23/pom.xml 91985fc 
  
hadooplibs/hadoop-utils-0.23/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java
 46c2fbd 
  
hadooplibs/hadoop-utils-0.23/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java
 9a19770 
  hadooplibs/hadoop-utils-1/pom.xml c81ea91 
  hadooplibs/hadoop-utils-2/pom.xml 2d3dbdd 
  hadooplibs/hadoop-utils-3/pom.xml 7270bfd 
  hadooplibs/pom.xml e5ea2e7 
  hbaselibs/hbase-0.94/pom.xml a65904e 
  hbaselibs/pom.xml 24051c5 
  hcataloglibs/hive-hcatalog-0.13.1/pom.xml b738741 
  hcataloglibs/pom.xml a96806b 
  minitest/pom.xml 8f656d2 
  pom.xml 5507381 
  sharelib/distcp/pom.xml b788ed0 
  sharelib/hcatalog/pom.xml b934be1 
  sharelib/hive/pom.xml ea088ff 
  sharelib/hive2/pom.xml e4c2465 
  sharelib/oozie/pom.xml 053cf1c 
  sharelib/pig/pom.xml ce75c68 
  sharelib/spark/pom.xml 744c95c 
  sharelib/sqoop/pom.xml 09f0b66 
  sharelib/streaming/pom.xml 50d15f7 
  src/main/assemblies/distro.xml 8e2bbae 
  src/main/assemblies/hadooplib.xml b32ca82 
  src/main/assemblies/hadooplibs.xml 9884efa 
  src/main/assemblies/hbaselib.xml 6e4db33 
  src/main/assemblies/hbaselibs.xml eb71624 
  src/main/assemblies/hcataloglib.xml 1002308 
  src/main/assemblies/hcataloglibs.xml 3d32a8f 
  tools/pom.xml 80fa425 
  tools/src/main/bin/ooziedb.sh 68158d8 
  tools/src/main/java/org/apache/oozie/tools/OozieDBCLI.java ca80d35 
  webapp/pom.xml a410b10 
  zookeeper-security-tests/pom.xml b24ce33 

Diff: https://reviews.apache.org/r/22615/diff/


Testing
---

Clean build with different hive and hbase versions


Thanks,

shwethags

Re: Review Request 22615: OOZIE-1876 use pom properties rather than specific version numbers in the pom files of hbaselibs, hcataloglibs, sharelib, etc

[shwetha . gs]

> On Nov. 26, 2014, 1:58 p.m., Jaydeep Vishwakarma wrote:
> > pom.xml, line 1606
> > 
> >
> > oozie.hadoop-utils.version already defined in common property.

Profile specific override. Makes it more readable


> On Nov. 26, 2014, 1:58 p.m., Jaydeep Vishwakarma wrote:
> > pom.xml, line 466
> > 
> >
> > why hadoop tool have distcp version.

Because hadoop-tools has distcp


> On Nov. 26, 2014, 1:58 p.m., Jaydeep Vishwakarma wrote:
> > examples/pom.xml, line 45
> > 
> >
> > Scope test?

The scope is already test in the base pom


> On Nov. 26, 2014, 1:58 p.m., Jaydeep Vishwakarma wrote:
> > core/pom.xml, line 46
> > 
> >
> > I think we use this in only for test, so scope should be test.

The scope is already test in the base pom


- shwethags


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/22615/#review63106
---


On Nov. 26, 2014, 6:45 a.m., shwethags wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/22615/
> ---
> 
> (Updated Nov. 26, 2014, 6:45 a.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-1876
> https://issues.apache.org/jira/browse/OOZIE-1876
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> 1. hbase/hcatalog versions configurable
> 2. Include hcatalog libs in final distro
> 
> 
> Diffs
> -
> 
>   client/pom.xml 365276d 
>   client/src/main/bin/oozie 44943fa 
>   core/pom.xml ca40e2e 
>   distro/src/main/bin/oozied.sh 8eafc96 
>   docs/src/site/twiki/DG_QuickStart.twiki 526fbc3 
>   docs/src/site/twiki/ENG_Building.twiki 8d74b72 
>   examples/pom.xml 852d222 
>   hadooplibs/hadoop-0.23/pom.xml 6297dbd 
>   hadooplibs/hadoop-1/pom.xml f0a32a5 
>   hadooplibs/hadoop-2/pom.xml 32d4b6c 
>   hadooplibs/hadoop-3/pom.xml 4fd8f29 
>   hadooplibs/hadoop-auth-0.23/pom.xml 59cc7f1 
>   hadooplibs/hadoop-auth-1/pom.xml 7da7639 
>   hadooplibs/hadoop-auth-2/pom.xml cbdb523 
>   hadooplibs/hadoop-distcp-0.23/pom.xml 6e42986 
>   hadooplibs/hadoop-distcp-1/pom.xml 01eb123 
>   hadooplibs/hadoop-distcp-2/pom.xml 7971eb5 
>   hadooplibs/hadoop-distcp-3/pom.xml 598ee13 
>   hadooplibs/hadoop-test-0.23/pom.xml addcd11 
>   hadooplibs/hadoop-test-1/pom.xml baf5a4c 
>   hadooplibs/hadoop-test-2/pom.xml 027c813 
>   hadooplibs/hadoop-test-3/pom.xml 7176f51 
>   hadooplibs/hadoop-utils-0.23/pom.xml 91985fc 
>   
> hadooplibs/hadoop-utils-0.23/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java
>  46c2fbd 
>   
> hadooplibs/hadoop-utils-0.23/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java
>  9a19770 
>   hadooplibs/hadoop-utils-1/pom.xml c81ea91 
>   hadooplibs/hadoop-utils-2/pom.xml 2d3dbdd 
>   hadooplibs/hadoop-utils-3/pom.xml 7270bfd 
>   hadooplibs/pom.xml e5ea2e7 
>   hbaselibs/hbase-0.94/pom.xml a65904e 
>   hbaselibs/pom.xml 24051c5 
>   hcataloglibs/hive-hcatalog-0.13.1/pom.xml b738741 
>   hcataloglibs/pom.xml a96806b 
>   minitest/pom.xml 8f656d2 
>   pom.xml 5507381 
>   sharelib/distcp/pom.xml b788ed0 
>   sharelib/hcatalog/pom.xml b934be1 
>   sharelib/hive/pom.xml ea088ff 
>   sharelib/hive2/pom.xml e4c2465 
>   sharelib/oozie/pom.xml 053cf1c 
>   sharelib/pig/pom.xml ce75c68 
>   sharelib/spark/pom.xml 744c95c 
>   sharelib/sqoop/pom.xml 09f0b66 
>   sharelib/streaming/pom.xml 50d15f7 
>   src/main/assemblies/distro.xml 8e2bbae 
>   src/main/assemblies/hadooplib.xml b32ca82 
>   src/main/assemblies/hadooplibs.xml 9884efa 
>   src/main/assemblies/hbaselib.xml 6e4db33 
>   src/main/assemblies/hbaselibs.xml eb71624 
>   src/main/assemblies/hcataloglib.xml 1002308 
>   src/main/assemblies/hcataloglibs.xml 3d32a8f 
>   tools/pom.xml 80fa425 
>   tools/src/main/bin/ooziedb.sh 68158d8 
>   webapp/pom.xml a410b10 
>   zookeeper-security-tests/pom.xml b24ce33 
> 
> Diff: https://reviews.apache.org/r/22615/diff/
> 
> 
> Testing
> ---
> 
> Clean build with different hive and hbase versions
> 
> 
> Thanks,
> 
> shwethags
> 
>

Re: Review Request 22615: OOZIE-1876 use pom properties rather than specific version numbers in the pom files of hbaselibs, hcataloglibs, sharelib, etc

[shwetha . gs]

> On Nov. 26, 2014, 11:12 p.m., Robert Kanter wrote:
> > This is really good.  I think this will help make the build more flexible.  
> > And I know working on it was probably not the most fun thing to do :)
> > 
> > I took a quick look through the patch.  
> > In addition to my two comments on the patch, I have one other concern: the 
> > patch changes a lot of the sharelib pom files.  We need to make sure that 
> > either the sharelibs each contain the exact set of jars they did before or 
> > that any changes in jars do not break any actions.  The former is probably 
> > easier because you don't have to do lots of testing beyond making sure the 
> > jars are there.  Can you look into this?
> 
> Robert Kanter wrote:
> One other thing; I don't have time right now to try this out, but I'll 
> try to do it next week.

Ok. Will check the sharelib


> On Nov. 26, 2014, 11:12 p.m., Robert Kanter wrote:
> > pom.xml, line 94
> > 
> >
> > There is no hadoop-auth for Hadoop 1.x; IIRC, we currently bundle 
> > hadoop-auth from 2.0.2-alpha when compiling against Hadoop 1.x.  Have you 
> > tried compiling against Hadoop 1?
> > 
> > If there isn't a way to reconcile this, we may have to hold your patch 
> > back until we drop Hadoop 1 support (at least in trunk), which there has 
> > been talk of before.

hadoop-1 uses hadoop-core for auth. See hadoop-auth-1/pom.xml


> On Nov. 26, 2014, 11:12 p.m., Robert Kanter wrote:
> > tools/src/main/bin/ooziedb.sh, lines 44-52
> > 
> >
> > I think the 'oozie-setup.sh sharelib' command will need something like 
> > this too, right?

I think so. Will fix it


- shwethags


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/22615/#review63153
---


On Nov. 26, 2014, 6:45 a.m., shwethags wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/22615/
> ---
> 
> (Updated Nov. 26, 2014, 6:45 a.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-1876
> https://issues.apache.org/jira/browse/OOZIE-1876
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> 1. hbase/hcatalog versions configurable
> 2. Include hcatalog libs in final distro
> 
> 
> Diffs
> -
> 
>   client/pom.xml 365276d 
>   client/src/main/bin/oozie 44943fa 
>   core/pom.xml ca40e2e 
>   distro/src/main/bin/oozied.sh 8eafc96 
>   docs/src/site/twiki/DG_QuickStart.twiki 526fbc3 
>   docs/src/site/twiki/ENG_Building.twiki 8d74b72 
>   examples/pom.xml 852d222 
>   hadooplibs/hadoop-0.23/pom.xml 6297dbd 
>   hadooplibs/hadoop-1/pom.xml f0a32a5 
>   hadooplibs/hadoop-2/pom.xml 32d4b6c 
>   hadooplibs/hadoop-3/pom.xml 4fd8f29 
>   hadooplibs/hadoop-auth-0.23/pom.xml 59cc7f1 
>   hadooplibs/hadoop-auth-1/pom.xml 7da7639 
>   hadooplibs/hadoop-auth-2/pom.xml cbdb523 
>   hadooplibs/hadoop-distcp-0.23/pom.xml 6e42986 
>   hadooplibs/hadoop-distcp-1/pom.xml 01eb123 
>   hadooplibs/hadoop-distcp-2/pom.xml 7971eb5 
>   hadooplibs/hadoop-distcp-3/pom.xml 598ee13 
>   hadooplibs/hadoop-test-0.23/pom.xml addcd11 
>   hadooplibs/hadoop-test-1/pom.xml baf5a4c 
>   hadooplibs/hadoop-test-2/pom.xml 027c813 
>   hadooplibs/hadoop-test-3/pom.xml 7176f51 
>   hadooplibs/hadoop-utils-0.23/pom.xml 91985fc 
>   
> hadooplibs/hadoop-utils-0.23/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java
>  46c2fbd 
>   
> hadooplibs/hadoop-utils-0.23/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java
>  9a19770 
>   hadooplibs/hadoop-utils-1/pom.xml c81ea91 
>   hadooplibs/hadoop-utils-2/pom.xml 2d3dbdd 
>   hadooplibs/hadoop-utils-3/pom.xml 7270bfd 
>   hadooplibs/pom.xml e5ea2e7 
>   hbaselibs/hbase-0.94/pom.xml a65904e 
>   hbaselibs/pom.xml 24051c5 
>   hcataloglibs/hive-hcatalog-0.13.1/pom.xml b738741 
>   hcataloglibs/pom.xml a96806b 
>   minitest/pom.xml 8f656d2 
>   pom.xml 5507381 
>   sharelib/distcp/pom.xml b788ed0 
>   sharelib/hcatalog/pom.xml b934be1 
>   sharelib/hive/pom.xml ea088ff 
>   sharelib/hive2/pom.xml e4c2465 
>   sharelib/oozie/pom.xml 053cf1c 
>   sharelib/pig/pom.xml ce75c68 
>   sharelib/spark/pom.xml 744c95c 
>   sharelib/sqoop/pom.xml 09f0b66 
>   sharelib/streaming/pom.xml 50d15f7 
>   src/main/assemblies/distro.xml 8e2bbae 
>   src/main/assemblies/hadooplib.xml b32ca82 
>   src/main/assemblies/hadooplibs.xml 9884efa 
>   src/main/assemblies/hbaselib.xml 6e4db33 
>   src/main/assemblies/hbaselibs.xml eb71624 
>   src/main/assemblies/hcataloglib.xml 1002308 
>   src/main/assemblies/hcataloglibs.xml 3d32a8f 
>   tools/pom.xml 80fa425 
>   tools/src/main/bin/ooziedb.sh 68158d8 
>   webapp/pom.xm

Re: Review Request 22615: OOZIE-1876 use pom properties rather than specific version numbers in the pom files of hbaselibs, hcataloglibs, sharelib, etc

[shwetha . gs]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/22615/
---

(Updated Nov. 26, 2014, 6:45 a.m.)


Review request for oozie.


Bugs: OOZIE-1876
https://issues.apache.org/jira/browse/OOZIE-1876


Repository: oozie-git


Description
---

1. hbase/hcatalog versions configurable
2. Include hcatalog libs in final distro


Diffs (updated)
-

  client/pom.xml 365276d 
  client/src/main/bin/oozie 44943fa 
  core/pom.xml ca40e2e 
  distro/src/main/bin/oozied.sh 8eafc96 
  docs/src/site/twiki/DG_QuickStart.twiki 526fbc3 
  docs/src/site/twiki/ENG_Building.twiki 8d74b72 
  examples/pom.xml 852d222 
  hadooplibs/hadoop-0.23/pom.xml 6297dbd 
  hadooplibs/hadoop-1/pom.xml f0a32a5 
  hadooplibs/hadoop-2/pom.xml 32d4b6c 
  hadooplibs/hadoop-3/pom.xml 4fd8f29 
  hadooplibs/hadoop-auth-0.23/pom.xml 59cc7f1 
  hadooplibs/hadoop-auth-1/pom.xml 7da7639 
  hadooplibs/hadoop-auth-2/pom.xml cbdb523 
  hadooplibs/hadoop-distcp-0.23/pom.xml 6e42986 
  hadooplibs/hadoop-distcp-1/pom.xml 01eb123 
  hadooplibs/hadoop-distcp-2/pom.xml 7971eb5 
  hadooplibs/hadoop-distcp-3/pom.xml 598ee13 
  hadooplibs/hadoop-test-0.23/pom.xml addcd11 
  hadooplibs/hadoop-test-1/pom.xml baf5a4c 
  hadooplibs/hadoop-test-2/pom.xml 027c813 
  hadooplibs/hadoop-test-3/pom.xml 7176f51 
  hadooplibs/hadoop-utils-0.23/pom.xml 91985fc 
  
hadooplibs/hadoop-utils-0.23/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java
 46c2fbd 
  
hadooplibs/hadoop-utils-0.23/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java
 9a19770 
  hadooplibs/hadoop-utils-1/pom.xml c81ea91 
  hadooplibs/hadoop-utils-2/pom.xml 2d3dbdd 
  hadooplibs/hadoop-utils-3/pom.xml 7270bfd 
  hadooplibs/pom.xml e5ea2e7 
  hbaselibs/hbase-0.94/pom.xml a65904e 
  hbaselibs/pom.xml 24051c5 
  hcataloglibs/hive-hcatalog-0.13.1/pom.xml b738741 
  hcataloglibs/pom.xml a96806b 
  minitest/pom.xml 8f656d2 
  pom.xml 5507381 
  sharelib/distcp/pom.xml b788ed0 
  sharelib/hcatalog/pom.xml b934be1 
  sharelib/hive/pom.xml ea088ff 
  sharelib/hive2/pom.xml e4c2465 
  sharelib/oozie/pom.xml 053cf1c 
  sharelib/pig/pom.xml ce75c68 
  sharelib/spark/pom.xml 744c95c 
  sharelib/sqoop/pom.xml 09f0b66 
  sharelib/streaming/pom.xml 50d15f7 
  src/main/assemblies/distro.xml 8e2bbae 
  src/main/assemblies/hadooplib.xml b32ca82 
  src/main/assemblies/hadooplibs.xml 9884efa 
  src/main/assemblies/hbaselib.xml 6e4db33 
  src/main/assemblies/hbaselibs.xml eb71624 
  src/main/assemblies/hcataloglib.xml 1002308 
  src/main/assemblies/hcataloglibs.xml 3d32a8f 
  tools/pom.xml 80fa425 
  tools/src/main/bin/ooziedb.sh 68158d8 
  webapp/pom.xml a410b10 
  zookeeper-security-tests/pom.xml b24ce33 

Diff: https://reviews.apache.org/r/22615/diff/


Testing
---

Clean build with different hive and hbase versions


Thanks,

shwethags

Re: Review Request 22615: OOZIE-1876 use pom properties rather than specific version numbers in the pom files of hbaselibs, hcataloglibs, sharelib, etc

[shwetha . gs]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/22615/
---

(Updated Nov. 26, 2014, 6:43 a.m.)


Review request for oozie.


Bugs: OOZIE-1876
https://issues.apache.org/jira/browse/OOZIE-1876


Repository: oozie-git


Description
---

1. hbase/hcatalog versions configurable
2. Include hcatalog libs in final distro


Diffs (updated)
-

  client/pom.xml 365276d 
  client/src/main/bin/oozie 44943fa 
  core/pom.xml ca40e2e 
  core/src/main/java/org/apache/oozie/service/ShareLibService.java bb0c7ed 
  distro/src/main/bin/oozied.sh 8eafc96 
  docs/src/site/twiki/DG_QuickStart.twiki 526fbc3 
  docs/src/site/twiki/ENG_Building.twiki 8d74b72 
  examples/pom.xml 852d222 
  hadooplibs/hadoop-0.23/pom.xml 6297dbd 
  hadooplibs/hadoop-1/pom.xml f0a32a5 
  hadooplibs/hadoop-2/pom.xml 32d4b6c 
  hadooplibs/hadoop-3/pom.xml 4fd8f29 
  hadooplibs/hadoop-auth-0.23/pom.xml 59cc7f1 
  hadooplibs/hadoop-auth-1/pom.xml 7da7639 
  hadooplibs/hadoop-auth-2/pom.xml cbdb523 
  hadooplibs/hadoop-distcp-0.23/pom.xml 6e42986 
  hadooplibs/hadoop-distcp-1/pom.xml 01eb123 
  hadooplibs/hadoop-distcp-2/pom.xml 7971eb5 
  hadooplibs/hadoop-distcp-3/pom.xml 598ee13 
  hadooplibs/hadoop-test-0.23/pom.xml addcd11 
  hadooplibs/hadoop-test-1/pom.xml baf5a4c 
  hadooplibs/hadoop-test-2/pom.xml 027c813 
  hadooplibs/hadoop-test-3/pom.xml 7176f51 
  hadooplibs/hadoop-utils-0.23/pom.xml 91985fc 
  
hadooplibs/hadoop-utils-0.23/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java
 46c2fbd 
  
hadooplibs/hadoop-utils-0.23/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java
 9a19770 
  hadooplibs/hadoop-utils-1/pom.xml c81ea91 
  hadooplibs/hadoop-utils-2/pom.xml 2d3dbdd 
  hadooplibs/hadoop-utils-3/pom.xml 7270bfd 
  hadooplibs/pom.xml e5ea2e7 
  hbaselibs/hbase-0.94/pom.xml a65904e 
  hbaselibs/pom.xml 24051c5 
  hcataloglibs/hive-hcatalog-0.13.1/pom.xml b738741 
  hcataloglibs/pom.xml a96806b 
  minitest/pom.xml 8f656d2 
  pom.xml 5507381 
  sharelib/distcp/pom.xml b788ed0 
  sharelib/hcatalog/pom.xml b934be1 
  sharelib/hive/pom.xml ea088ff 
  sharelib/hive2/pom.xml e4c2465 
  sharelib/oozie/pom.xml 053cf1c 
  sharelib/pig/pom.xml ce75c68 
  sharelib/spark/pom.xml 744c95c 
  sharelib/sqoop/pom.xml 09f0b66 
  sharelib/streaming/pom.xml 50d15f7 
  src/main/assemblies/distro.xml 8e2bbae 
  src/main/assemblies/hadooplib.xml b32ca82 
  src/main/assemblies/hadooplibs.xml 9884efa 
  src/main/assemblies/hbaselib.xml 6e4db33 
  src/main/assemblies/hbaselibs.xml eb71624 
  src/main/assemblies/hcataloglib.xml 1002308 
  src/main/assemblies/hcataloglibs.xml 3d32a8f 
  tools/pom.xml 80fa425 
  tools/src/main/bin/ooziedb.sh 68158d8 
  webapp/pom.xml a410b10 
  zookeeper-security-tests/pom.xml b24ce33 

Diff: https://reviews.apache.org/r/22615/diff/


Testing
---

Clean build with different hive and hbase versions


Thanks,

shwethags

Re: Review Request 22615: OOZIE-1876 use pom properties rather than specific version numbers in the pom files of hbaselibs, hcataloglibs, sharelib, etc

[shwetha . gs]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/22615/
---

(Updated Nov. 26, 2014, 6:42 a.m.)


Review request for oozie.


Changes
---

Re-based after spark changed


Bugs: OOZIE-1876
https://issues.apache.org/jira/browse/OOZIE-1876


Repository: oozie-git


Description
---

1. hbase/hcatalog versions configurable
2. Include hcatalog libs in final distro


Diffs (updated)
-

  client/pom.xml 365276d 
  client/src/main/bin/oozie 44943fa 
  core/pom.xml ca40e2e 
  core/src/main/java/org/apache/oozie/service/ShareLibService.java bb0c7ed 
  distro/src/main/bin/oozied.sh 8eafc96 
  docs/src/site/twiki/DG_QuickStart.twiki 526fbc3 
  docs/src/site/twiki/ENG_Building.twiki 8d74b72 
  examples/pom.xml 852d222 
  hadooplibs/hadoop-0.23/pom.xml 6297dbd 
  hadooplibs/hadoop-1/pom.xml f0a32a5 
  hadooplibs/hadoop-2/pom.xml 32d4b6c 
  hadooplibs/hadoop-3/pom.xml 4fd8f29 
  hadooplibs/hadoop-auth-0.23/pom.xml 59cc7f1 
  hadooplibs/hadoop-auth-1/pom.xml 7da7639 
  hadooplibs/hadoop-auth-2/pom.xml cbdb523 
  hadooplibs/hadoop-distcp-0.23/pom.xml 6e42986 
  hadooplibs/hadoop-distcp-1/pom.xml 01eb123 
  hadooplibs/hadoop-distcp-2/pom.xml 7971eb5 
  hadooplibs/hadoop-distcp-3/pom.xml 598ee13 
  hadooplibs/hadoop-test-0.23/pom.xml addcd11 
  hadooplibs/hadoop-test-1/pom.xml baf5a4c 
  hadooplibs/hadoop-test-2/pom.xml 027c813 
  hadooplibs/hadoop-test-3/pom.xml 7176f51 
  hadooplibs/hadoop-utils-0.23/pom.xml 91985fc 
  
hadooplibs/hadoop-utils-0.23/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java
 46c2fbd 
  
hadooplibs/hadoop-utils-0.23/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java
 9a19770 
  hadooplibs/hadoop-utils-1/pom.xml c81ea91 
  hadooplibs/hadoop-utils-2/pom.xml 2d3dbdd 
  hadooplibs/hadoop-utils-3/pom.xml 7270bfd 
  hadooplibs/pom.xml e5ea2e7 
  hbaselibs/hbase-0.94/pom.xml a65904e 
  hbaselibs/pom.xml 24051c5 
  hcataloglibs/hive-hcatalog-0.13.1/pom.xml b738741 
  hcataloglibs/pom.xml a96806b 
  minitest/pom.xml 8f656d2 
  pom.xml 5507381 
  sharelib/distcp/pom.xml b788ed0 
  sharelib/hcatalog/pom.xml b934be1 
  sharelib/hive/pom.xml ea088ff 
  sharelib/hive2/pom.xml e4c2465 
  sharelib/oozie/pom.xml 053cf1c 
  sharelib/pig/pom.xml ce75c68 
  sharelib/spark/pom.xml 744c95c 
  sharelib/sqoop/pom.xml 09f0b66 
  sharelib/streaming/pom.xml 50d15f7 
  src/main/assemblies/distro.xml 8e2bbae 
  src/main/assemblies/hadooplib.xml b32ca82 
  src/main/assemblies/hadooplibs.xml 9884efa 
  src/main/assemblies/hbaselib.xml 6e4db33 
  src/main/assemblies/hbaselibs.xml eb71624 
  src/main/assemblies/hcataloglib.xml 1002308 
  src/main/assemblies/hcataloglibs.xml 3d32a8f 
  tools/pom.xml 80fa425 
  tools/src/main/bin/ooziedb.sh 68158d8 
  webapp/pom.xml a410b10 
  zookeeper-security-tests/pom.xml b24ce33 

Diff: https://reviews.apache.org/r/22615/diff/


Testing
---

Clean build with different hive and hbase versions


Thanks,

shwethags

Re: Review Request 22615: OOZIE-1876 use pom properties rather than specific version numbers in the pom files of hbaselibs, hcataloglibs, sharelib, etc

[shwetha . gs]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/22615/
---

(Updated Nov. 20, 2014, 11:59 a.m.)


Review request for oozie.


Changes
---

Finally, here is a working patch

Here are the changes:
1. All the versions are configurable - hadoop, hive, distcp, auth
2. oozie.war bundles all the required jars - required hadoop and hcat 
dependencies. (Reasoning: Currently, we need to add hadoop and hcat jars at 
runtime in libext. Since we won't know which are the exact jars required, we 
end up adding everything in hadoop and hcat classpath. In newer versions of 
hadoop and hive, there are so many jars that the final oozie application has 
multiple versions of the same jar and creates conflicts)
3. ooze-start without ooze-setup does setup automatically
4. Removed packaging jars in lib and libtools. Instead, picks up the jars by 
exploding oozie.war

Testing:
1. Compiled using different combinations of hadoop profile and version
2. Verified few workflows with oozie setup with hadoop-2.5.0


Bugs: OOZIE-1876
https://issues.apache.org/jira/browse/OOZIE-1876


Repository: oozie-git


Description
---

1. hbase/hcatalog versions configurable
2. Include hcatalog libs in final distro


Diffs (updated)
-

  client/pom.xml 365276d 
  client/src/main/bin/oozie 44943fa 
  core/pom.xml ca40e2e 
  distro/src/main/bin/oozied.sh 8eafc96 
  docs/src/site/twiki/DG_QuickStart.twiki 526fbc3 
  docs/src/site/twiki/ENG_Building.twiki 8d74b72 
  examples/pom.xml 852d222 
  hadooplibs/hadoop-0.23/pom.xml 6297dbd 
  hadooplibs/hadoop-1/pom.xml f0a32a5 
  hadooplibs/hadoop-2/pom.xml 32d4b6c 
  hadooplibs/hadoop-3/pom.xml 4fd8f29 
  hadooplibs/hadoop-auth-0.23/pom.xml 59cc7f1 
  hadooplibs/hadoop-auth-1/pom.xml 7da7639 
  hadooplibs/hadoop-auth-2/pom.xml cbdb523 
  hadooplibs/hadoop-distcp-0.23/pom.xml 6e42986 
  hadooplibs/hadoop-distcp-1/pom.xml 01eb123 
  hadooplibs/hadoop-distcp-2/pom.xml 7971eb5 
  hadooplibs/hadoop-distcp-3/pom.xml 598ee13 
  hadooplibs/hadoop-test-0.23/pom.xml addcd11 
  hadooplibs/hadoop-test-1/pom.xml baf5a4c 
  hadooplibs/hadoop-test-2/pom.xml 027c813 
  hadooplibs/hadoop-test-3/pom.xml 7176f51 
  hadooplibs/hadoop-utils-0.23/pom.xml 91985fc 
  
hadooplibs/hadoop-utils-0.23/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java
 46c2fbd 
  
hadooplibs/hadoop-utils-0.23/src/main/java/org/apache/oozie/hadoop/utils/HadoopShims.java
 9a19770 
  hadooplibs/hadoop-utils-1/pom.xml c81ea91 
  hadooplibs/hadoop-utils-2/pom.xml 2d3dbdd 
  hadooplibs/hadoop-utils-3/pom.xml 7270bfd 
  hadooplibs/pom.xml e5ea2e7 
  hbaselibs/hbase-0.94/pom.xml a65904e 
  hbaselibs/pom.xml 24051c5 
  hcataloglibs/hive-hcatalog-0.13.1/pom.xml b738741 
  hcataloglibs/pom.xml a96806b 
  minitest/pom.xml 8f656d2 
  pom.xml 1e79186 
  sharelib/distcp/pom.xml b788ed0 
  sharelib/hcatalog/pom.xml b934be1 
  sharelib/hive/pom.xml ea088ff 
  sharelib/hive2/pom.xml e4c2465 
  sharelib/oozie/pom.xml 053cf1c 
  sharelib/pig/pom.xml ce75c68 
  sharelib/sqoop/pom.xml 09f0b66 
  sharelib/streaming/pom.xml 50d15f7 
  src/main/assemblies/distro.xml 8e2bbae 
  src/main/assemblies/hadooplib.xml b32ca82 
  src/main/assemblies/hadooplibs.xml 9884efa 
  src/main/assemblies/hbaselib.xml 6e4db33 
  src/main/assemblies/hbaselibs.xml eb71624 
  src/main/assemblies/hcataloglib.xml 1002308 
  src/main/assemblies/hcataloglibs.xml 3d32a8f 
  tools/pom.xml 80fa425 
  tools/src/main/bin/ooziedb.sh 68158d8 
  webapp/pom.xml 35776c5 
  zookeeper-security-tests/pom.xml b24ce33 

Diff: https://reviews.apache.org/r/22615/diff/


Testing
---

Clean build with different hive and hbase versions


Thanks,

shwethags

Re: VOTE Release Oozie 4.1.0 (candidate 1)

[Shwetha GS]

+1

On Fri, Nov 14, 2014 at 6:49 AM, bowen zhang <
bowenzhang...@yahoo.com.invalid> wrote:

> Hi,
>
> I have created a build for Oozie 4.1.0, candidate 1.
>
> Keys to verify the signature of the release artifact are available at
>
>   http://www.apache.org/dist/oozie/KEYS
>
> Please download, test, and try it out:
>
>   http://people.apache.org/~bzhang/oozie-4.1.0-rc1
>
> The release, md5 signature, gpg signature, and rat report can all
> be found at the above address.
>
> Vote closes on Monday EOD, the 17th.
>
> Bowen
>

-- 
_
The information contained in this communication is intended solely for the 
use of the individual or entity to whom it is addressed and others 
authorized to receive it. It may contain confidential or legally privileged 
information. If you are not the intended recipient you are hereby notified 
that any disclosure, copying, distribution or taking any action in reliance 
on the contents of this information is strictly prohibited and may be 
unlawful. If you have received this communication in error, please notify 
us immediately by responding to this email and then delete it from your 
system. The firm is neither liable for the proper and complete transmission 
of the information contained in this communication nor for any delay in its 
receipt.

Re: Problem with coord:current?

[Shwetha GS]

The ds initial instance is "2013-01-01T00:01Z"(Note 1st minute of hour).
So, the instance before nominal time 2014-09-03T*09*:00Z is 2014/09/03/*08.
If you change the initial instance to *"2013-01-01T00:00Z", the resolved
path will be for 2014/09/03/*09*

*-Shwetha*

On Wed, Sep 24, 2014 at 5:11 AM, Robert Kanter  wrote:

> Hi all,
>
> Suppose I submit the attached Coordinator job.  The important parts to
> note are that the start time is 09/03/2014 at 9:00am, the URI template ends
> with /${YEAR}/${MONTH}/${DAY}/${HOUR}, and the input-event is using
> ${coord:current(0)}.
>
> When you submit this, current(0) seems to be incorrect:
> ID : 000-140923162340061-oozie-rkan-C@1
>
> --
> Action Number: 1
> Console URL  : -
> Error Code   : -
> Error Message: -
> External ID  : -
> External Status  : -
> Job ID   : 000-140923162340061-oozie-rkan-C
> Tracker URI  : -
> Created  : 2014-09-23 23:23 GMT
> Nominal Time : 2014-09-03 *09*:00 GMT
> Status   : WAITING
> Last Modified: 2014-09-23 23:23 GMT
> First Missing Dependency :
> hdfs://localhost:8020/user/rkanter/path_coord/input-data/2014/09/03/*08*
> /_SUCCESS
>
> --
> *Notice that the first missing dependency ends with 08 instead of 09, even
> though the nominal time is 9am!*
>
> I was poking through the code and it looks like the problem may be in
> CoordELFunctions.getCurrentInstance(...) which subtracts once frequency
> amount from the computed current instance.  If I remove that code, it
> appears to behave correctly:
> diff --git
> a/core/src/main/java/org/apache/oozie/coord/CoordELFunctions.java
> b/core/src/main/java/org/apache/oozie/coord/CoordELFunctions.java
> index 3bb191e..02f4166 100644
> --- a/core/src/main/java/org/apache/oozie/coord/CoordELFunctions.java
> +++ b/core/src/main/java/org/apache/oozie/coord/CoordELFunctions.java
> @@ -1343,8 +1343,6 @@ public class CoordELFunctions {
>  current.add(dsTimeUnit.getCalendarUnit(), dsFreq);
>  instanceCount[0]++;
>  }
> -current.add(dsTimeUnit.getCalendarUnit(), -dsFreq);
> -instanceCount[0]--;
>  return current;
>  }
> When I run the original code through a debugger, current is set to 8am;
> my modified code has 9am.
>
> Is there something I'm missing?  I feel like we would have noticed this by
> now because current is pretty common...
>
> This code was greatly changed a few months ago by OOZIE-1709, but the old
> code did the same thing were it would subtract at the end and had 8am
> instead of 9am.
>
>
> thanks
> - Robert
>

-- 
_
The information contained in this communication is intended solely for the 
use of the individual or entity to whom it is addressed and others 
authorized to receive it. It may contain confidential or legally privileged 
information. If you are not the intended recipient you are hereby notified 
that any disclosure, copying, distribution or taking any action in reliance 
on the contents of this information is strictly prohibited and may be 
unlawful. If you have received this communication in error, please notify 
us immediately by responding to this email and then delete it from your 
system. The firm is neither liable for the proper and complete transmission 
of the information contained in this communication nor for any delay in its 
receipt.

Re: Review Request 24165: OOZIE-1954: Add a way for the MapReduce action to be configured by Java code

[shwetha . gs]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24165/#review53508
---

Ship it!


Ship It!

- shwethags


On Sept. 15, 2014, 6:14 p.m., Robert Kanter wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/24165/
> ---
> 
> (Updated Sept. 15, 2014, 6:14 p.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-1954
> https://issues.apache.org/jira/browse/OOZIE-1954
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> See https://issues.apache.org/jira/browse/OOZIE-1954
> 
> 
> Diffs
> -
> 
>   client/src/main/resources/oozie-workflow-0.5.xsd 6620a4e 
>   core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 
> 53f979c 
>   core/src/test/java/org/apache/oozie/action/hadoop/MapperReducerForTest.java 
> d89990c 
>   docs/src/site/twiki/WorkflowFunctionalSpec.twiki 3319bcc 
>   examples/src/main/apps/map-reduce/job-with-config-class.properties 
> PRE-CREATION 
>   examples/src/main/apps/map-reduce/job.properties 7b7a24c 
>   examples/src/main/apps/map-reduce/workflow-with-config-class.xml 
> PRE-CREATION 
>   
> examples/src/main/java/org/apache/oozie/example/SampleOozieActionConfigurator.java
>  PRE-CREATION 
>   
> sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMain.java 
> ed5b88c 
>   
> sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMapper.java
>  ada0706 
>   
> sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/MapReduceMain.java
>  1a9a194 
>   
> sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/OozieActionConfigurator.java
>  PRE-CREATION 
>   
> sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/OozieActionConfiguratorException.java
>  PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PipesMain.java 
> ca32b5f 
>   
> sharelib/streaming/src/test/java/org/apache/oozie/action/hadoop/OozieActionConfiguratorForTest.java
>  PRE-CREATION 
>   
> sharelib/streaming/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java
>  d43ddca 
> 
> Diff: https://reviews.apache.org/r/24165/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Robert Kanter
> 
>

Re: Review Request 24165: OOZIE-1954: Add a way for the MapReduce action to be configured by Java code

[shwetha . gs]

> On Sept. 12, 2014, 11:18 a.m., shwethags wrote:
> > docs/src/site/twiki/WorkflowFunctionalSpec.twiki, line 634
> > 
> >
> > Will the JobConf have all the workflow params? Else, it may not be 
> > useful. For example, setting input path and output path depends on workflow 
> > params
> 
> Robert Kanter wrote:
> You mean the stuff that gets set in job.properties?  I'll look if there's 
> a good way to include them.
> 
> Robert Kanter wrote:
> I don't think there's a good way to do this.  The OozieActionConfigurator 
> gets called from the launcher job, which doesn't have access to the 
> parameters.  I think it should be okay not to include these directly because 
> (a) the actionConf will contain resolved values already (i.e. if the user 
> set "mapred.input.path" to "/user/${user.name}/foo" in the action's 
>  section, that would already be resolved to 
> "/user/rkanter/foo" in the OozieActionConfigurator
> (b) if the user really wants to propagate a value to be used in this 
> code, they can simply set it in the action's  section and read 
> it from the actionConf
> 
> Makes sense?
> 
> Robert Kanter wrote:
> I wrote this patch so long ago, that I forgot that in the example 
> examples/src/main/apps/map-reduce/workflow-with-config-class.xml I actually 
> do this

Can you add this to the twiki doc and the interface doc? Looks good otherwise


- shwethags


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24165/#review53165
---


On Sept. 12, 2014, 10:13 p.m., Robert Kanter wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/24165/
> ---
> 
> (Updated Sept. 12, 2014, 10:13 p.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-1954
> https://issues.apache.org/jira/browse/OOZIE-1954
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> See https://issues.apache.org/jira/browse/OOZIE-1954
> 
> 
> Diffs
> -
> 
>   client/src/main/resources/oozie-workflow-0.5.xsd 6620a4e 
>   core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 
> 53f979c 
>   core/src/test/java/org/apache/oozie/action/hadoop/MapperReducerForTest.java 
> d89990c 
>   docs/src/site/twiki/WorkflowFunctionalSpec.twiki 3319bcc 
>   examples/src/main/apps/map-reduce/job-with-config-class.properties 
> PRE-CREATION 
>   examples/src/main/apps/map-reduce/job.properties 7b7a24c 
>   examples/src/main/apps/map-reduce/workflow-with-config-class.xml 
> PRE-CREATION 
>   
> examples/src/main/java/org/apache/oozie/example/SampleOozieActionConfigurator.java
>  PRE-CREATION 
>   
> sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMain.java 
> ed5b88c 
>   
> sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMapper.java
>  ada0706 
>   
> sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/MapReduceMain.java
>  1a9a194 
>   
> sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/OozieActionConfigurator.java
>  PRE-CREATION 
>   
> sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/OozieActionConfiguratorException.java
>  PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PipesMain.java 
> ca32b5f 
>   
> sharelib/streaming/src/test/java/org/apache/oozie/action/hadoop/OozieActionConfiguratorForTest.java
>  PRE-CREATION 
>   
> sharelib/streaming/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java
>  d43ddca 
> 
> Diff: https://reviews.apache.org/r/24165/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Robert Kanter
> 
>

Re: Review Request 24165: OOZIE-1954: Add a way for the MapReduce action to be configured by Java code

[shwetha . gs]

> On Sept. 9, 2014, 9:24 a.m., shwethags wrote:
> > docs/src/site/twiki/WorkflowFunctionalSpec.twiki, line 605
> > 
> >
> > Instead of depending on oozie artifact for the interface(which is 
> > difficult to get given that oozie artifacts are not available in any mvn 
> > repo), can the interface be org.apache.hadoop.util.Tool. The user provided 
> > run() can update the Conf. 
> > 
> > run() doesn't sound like setup, but serves the purpose with conf.
> 
> Robert Kanter wrote:
> Using Tool gives us less control if we want to make any changes (e.g. add 
> new methods, etc) in the future, so I'd prefer to use an Oozie-specific 
> interface.  It's also more clear what this is for.
> 
> I think the real solution for that difficulting is to publish artifacts, 
> which many users have been asking for, and everyone seems to agree that we 
> should do.  I'm hoping that we'll publish starting with the 4.1 release.
> 
> Robert Kanter wrote:
> I've talked with Bowen offline and he's going to publish the 4.1  
> (OOZIE-1842).  And by the time we actually release OOZIE-1954, publishing to 
> maven should be part of the release process.
> 
> shwethags wrote:
> I am fine with it if the oozie artifacts are available. 
> 
> If we move the interface to oozie-client, then the launcher job will need 
> oozie-client jar as well?
> 
> Robert Kanter wrote:
> Correct.  I think putting it in the oozie-sharelib (or perhaps a new 
> artifact?) makes the most sense.

If all the oozie artifacts are in mvn repo, we can go with oozie-sharelib itself


- shwethags


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24165/#review52696
---


On July 31, 2014, 10:33 p.m., Robert Kanter wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/24165/
> ---
> 
> (Updated July 31, 2014, 10:33 p.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-1954
> https://issues.apache.org/jira/browse/OOZIE-1954
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> See https://issues.apache.org/jira/browse/OOZIE-1954
> 
> 
> Diffs
> -
> 
>   client/src/main/resources/oozie-workflow-0.5.xsd 6620a4e 
>   core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 
> 94b55cf 
>   core/src/test/java/org/apache/oozie/action/hadoop/MapperReducerForTest.java 
> 7fe1f42 
>   docs/src/site/twiki/WorkflowFunctionalSpec.twiki 3319bcc 
>   examples/src/main/apps/map-reduce/job-with-config-class.properties 
> PRE-CREATION 
>   examples/src/main/apps/map-reduce/job.properties 7b7a24c 
>   examples/src/main/apps/map-reduce/workflow-with-config-class.xml 
> PRE-CREATION 
>   
> examples/src/main/java/org/apache/oozie/example/SampleOozieActionConfigurator.java
>  PRE-CREATION 
>   
> sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMain.java 
> 2cb690f 
>   
> sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMapper.java
>  322f498 
>   
> sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/MapReduceMain.java
>  f417a78 
>   
> sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/OozieActionConfigurator.java
>  PRE-CREATION 
>   
> sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/OozieActionConfiguratorException.java
>  PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PipesMain.java 
> 67a2a43 
>   
> sharelib/streaming/src/test/java/org/apache/oozie/action/hadoop/OozieActionConfiguratorForTest.java
>  PRE-CREATION 
>   
> sharelib/streaming/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java
>  e78c98a 
>   
> sharelib/streaming/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutorUberJar.java
>  bcc22f2 
> 
> Diff: https://reviews.apache.org/r/24165/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Robert Kanter
> 
>

Re: Review Request 24165: OOZIE-1954: Add a way for the MapReduce action to be configured by Java code

[shwetha . gs]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24165/#review53165
---



docs/src/site/twiki/WorkflowFunctionalSpec.twiki


rename to MapredActionConfigurator?



docs/src/site/twiki/WorkflowFunctionalSpec.twiki


rename to configure?



docs/src/site/twiki/WorkflowFunctionalSpec.twiki


Will the JobConf have all the workflow params? Else, it may not be useful. 
For example, setting input path and output path depends on workflow params



docs/src/site/twiki/WorkflowFunctionalSpec.twiki


the jar can be in workflow lib path or need to be specified in  
option of action?


- shwethags


On July 31, 2014, 10:33 p.m., Robert Kanter wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/24165/
> ---
> 
> (Updated July 31, 2014, 10:33 p.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-1954
> https://issues.apache.org/jira/browse/OOZIE-1954
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> See https://issues.apache.org/jira/browse/OOZIE-1954
> 
> 
> Diffs
> -
> 
>   client/src/main/resources/oozie-workflow-0.5.xsd 6620a4e 
>   core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 
> 94b55cf 
>   core/src/test/java/org/apache/oozie/action/hadoop/MapperReducerForTest.java 
> 7fe1f42 
>   docs/src/site/twiki/WorkflowFunctionalSpec.twiki 3319bcc 
>   examples/src/main/apps/map-reduce/job-with-config-class.properties 
> PRE-CREATION 
>   examples/src/main/apps/map-reduce/job.properties 7b7a24c 
>   examples/src/main/apps/map-reduce/workflow-with-config-class.xml 
> PRE-CREATION 
>   
> examples/src/main/java/org/apache/oozie/example/SampleOozieActionConfigurator.java
>  PRE-CREATION 
>   
> sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMain.java 
> 2cb690f 
>   
> sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMapper.java
>  322f498 
>   
> sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/MapReduceMain.java
>  f417a78 
>   
> sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/OozieActionConfigurator.java
>  PRE-CREATION 
>   
> sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/OozieActionConfiguratorException.java
>  PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PipesMain.java 
> 67a2a43 
>   
> sharelib/streaming/src/test/java/org/apache/oozie/action/hadoop/OozieActionConfiguratorForTest.java
>  PRE-CREATION 
>   
> sharelib/streaming/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java
>  e78c98a 
>   
> sharelib/streaming/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutorUberJar.java
>  bcc22f2 
> 
> Diff: https://reviews.apache.org/r/24165/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Robert Kanter
> 
>

Re: Review Request 24165: OOZIE-1954: Add a way for the MapReduce action to be configured by Java code

[shwetha . gs]

> On Sept. 9, 2014, 9:24 a.m., shwethags wrote:
> > docs/src/site/twiki/WorkflowFunctionalSpec.twiki, line 605
> > 
> >
> > Instead of depending on oozie artifact for the interface(which is 
> > difficult to get given that oozie artifacts are not available in any mvn 
> > repo), can the interface be org.apache.hadoop.util.Tool. The user provided 
> > run() can update the Conf. 
> > 
> > run() doesn't sound like setup, but serves the purpose with conf.
> 
> Robert Kanter wrote:
> Using Tool gives us less control if we want to make any changes (e.g. add 
> new methods, etc) in the future, so I'd prefer to use an Oozie-specific 
> interface.  It's also more clear what this is for.
> 
> I think the real solution for that difficulting is to publish artifacts, 
> which many users have been asking for, and everyone seems to agree that we 
> should do.  I'm hoping that we'll publish starting with the 4.1 release.
> 
> Robert Kanter wrote:
> I've talked with Bowen offline and he's going to publish the 4.1  
> (OOZIE-1842).  And by the time we actually release OOZIE-1954, publishing to 
> maven should be part of the release process.

I am fine with it if the oozie artifacts are available. 

If we move the interface to oozie-client, then the launcher job will need 
oozie-client jar as well?


- shwethags


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24165/#review52696
---


On July 31, 2014, 10:33 p.m., Robert Kanter wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/24165/
> ---
> 
> (Updated July 31, 2014, 10:33 p.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-1954
> https://issues.apache.org/jira/browse/OOZIE-1954
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> See https://issues.apache.org/jira/browse/OOZIE-1954
> 
> 
> Diffs
> -
> 
>   client/src/main/resources/oozie-workflow-0.5.xsd 6620a4e 
>   core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 
> 94b55cf 
>   core/src/test/java/org/apache/oozie/action/hadoop/MapperReducerForTest.java 
> 7fe1f42 
>   docs/src/site/twiki/WorkflowFunctionalSpec.twiki 3319bcc 
>   examples/src/main/apps/map-reduce/job-with-config-class.properties 
> PRE-CREATION 
>   examples/src/main/apps/map-reduce/job.properties 7b7a24c 
>   examples/src/main/apps/map-reduce/workflow-with-config-class.xml 
> PRE-CREATION 
>   
> examples/src/main/java/org/apache/oozie/example/SampleOozieActionConfigurator.java
>  PRE-CREATION 
>   
> sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMain.java 
> 2cb690f 
>   
> sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMapper.java
>  322f498 
>   
> sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/MapReduceMain.java
>  f417a78 
>   
> sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/OozieActionConfigurator.java
>  PRE-CREATION 
>   
> sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/OozieActionConfiguratorException.java
>  PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PipesMain.java 
> 67a2a43 
>   
> sharelib/streaming/src/test/java/org/apache/oozie/action/hadoop/OozieActionConfiguratorForTest.java
>  PRE-CREATION 
>   
> sharelib/streaming/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java
>  e78c98a 
>   
> sharelib/streaming/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutorUberJar.java
>  bcc22f2 
> 
> Diff: https://reviews.apache.org/r/24165/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Robert Kanter
> 
>

Re: Review Request 24299: OOZIE-1950 Coordinator job info should support timestamp (nominal time)

[shwetha . gs]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24299/
---

(Updated Sept. 10, 2014, 9:02 a.m.)


Review request for oozie.


Bugs: OOZIE-1950
https://issues.apache.org/jira/browse/OOZIE-1950


Repository: oozie-git


Description
---

Adds filter on nominal time in get coord actions


Diffs (updated)
-

  client/src/main/java/org/apache/oozie/client/OozieClient.java 363ebd2 
  core/src/main/java/org/apache/oozie/CoordinatorEngine.java 249ae68 
  core/src/main/java/org/apache/oozie/command/coord/CoordJobXCommand.java 
4d5ba71 
  
core/src/main/java/org/apache/oozie/executor/jpa/CoordJobGetActionsSubsetJPAExecutor.java
 cdd5000 
  core/src/main/java/org/apache/oozie/util/Pair.java PRE-CREATION 
  core/src/test/java/org/apache/oozie/TestCoordinatorEngine.java 93c9235 
  core/src/test/java/org/apache/oozie/TestCoordinatorEngineSimple.java 824eb80 
  
core/src/test/java/org/apache/oozie/executor/jpa/TestCoordJobGetActionsSubsetJPAExecutor.java
 d3d29aa 
  docs/src/site/twiki/DG_CommandLineTool.twiki 4a07711 

Diff: https://reviews.apache.org/r/24299/diff/


Testing
---

UTs


Thanks,

shwethags

Re: Review Request 24299: OOZIE-1950 Coordinator job info should support timestamp (nominal time)

[shwetha . gs]

> On Sept. 8, 2014, 6:50 p.m., Purshotam Shah wrote:
> > core/src/main/java/org/apache/oozie/util/Pair.java, line 73
> > 
> >
> > 1.
> > If first!=null and otherPair.first=null
> > second=null and otherPair.second=null.
> > 
> > Then this will return true.
> > 2.
> > If first=abc and otherPair.first=xyz
> > second=null and otherPair.second=null.
> > 
> > Then this will return true.
> > 
> > Is this ok?

I copied this from Hive.Pair and assumed that they must have written it right 
and didn't check. Yes, this will not work. Will fix


- shwethags


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24299/#review52619
---


On Sept. 8, 2014, 6:15 a.m., shwethags wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/24299/
> ---
> 
> (Updated Sept. 8, 2014, 6:15 a.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-1950
> https://issues.apache.org/jira/browse/OOZIE-1950
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> Adds filter on nominal time in get coord actions
> 
> 
> Diffs
> -
> 
>   client/src/main/java/org/apache/oozie/client/OozieClient.java 363ebd2 
>   core/src/main/java/org/apache/oozie/CoordinatorEngine.java 249ae68 
>   core/src/main/java/org/apache/oozie/command/coord/CoordJobXCommand.java 
> 4d5ba71 
>   
> core/src/main/java/org/apache/oozie/executor/jpa/CoordJobGetActionsSubsetJPAExecutor.java
>  cdd5000 
>   core/src/main/java/org/apache/oozie/util/Pair.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/TestCoordinatorEngine.java 93c9235 
>   core/src/test/java/org/apache/oozie/TestCoordinatorEngineSimple.java 
> 824eb80 
>   
> core/src/test/java/org/apache/oozie/executor/jpa/TestCoordJobGetActionsSubsetJPAExecutor.java
>  d3d29aa 
>   docs/src/site/twiki/DG_CommandLineTool.twiki 4a07711 
> 
> Diff: https://reviews.apache.org/r/24299/diff/
> 
> 
> Testing
> ---
> 
> UTs
> 
> 
> Thanks,
> 
> shwethags
> 
>

Re: Review Request 25041: OOZIE-1457: Create a Hive Server 2 action

[shwetha . gs]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25041/#review52828
---



core/pom.xml


even hbase credential requires hbase jar which is not included in oozie 
war. It depends on config, right? If credential is enabled in config, users can 
add the jar manually as well?


Minor comment about including hive-jdbc. Looks good otherwise. If you have 
tested end to end flow, good to go

- shwethags


On Sept. 9, 2014, 11:56 p.m., Robert Kanter wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/25041/
> ---
> 
> (Updated Sept. 9, 2014, 11:56 p.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-1457
> https://issues.apache.org/jira/browse/OOZIE-1457
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> See https://issues.apache.org/jira/browse/OOZIE-1457
> 
> 
> Diffs
> -
> 
>   client/src/main/java/org/apache/oozie/cli/OozieCLI.java 79a9b68 
>   client/src/main/resources/hive2-action-0.1.xsd PRE-CREATION 
>   core/pom.xml 59b6ecd 
>   core/src/main/conf/oozie-site.xml 32f87f5 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2ActionExecutor.java 
> PRE-CREATION 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2Credentials.java 
> PRE-CREATION 
>   core/src/test/java/org/apache/oozie/test/XTestCase.java 0dd7d71 
>   core/src/test/java/org/apache/oozie/test/hive/AbstractHiveService.java 
> PRE-CREATION 
>   core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java PRE-CREATION 
>   docs/src/site/twiki/DG_Hive2ActionExtension.twiki PRE-CREATION 
>   docs/src/site/twiki/DG_UnifiedCredentialsModule.twiki 7c52970 
>   docs/src/site/twiki/index.twiki f078bf5 
>   pom.xml 0b19431 
>   sharelib/hive2/pom.xml PRE-CREATION 
>   sharelib/hive2/src/main/java/org/apache/oozie/action/hadoop/Hive2Main.java 
> PRE-CREATION 
>   
> sharelib/hive2/src/test/java/org/apache/oozie/action/hadoop/TestHive2ActionExecutor.java
>  PRE-CREATION 
>   sharelib/pom.xml 1d6e4bd 
>   src/main/assemblies/sharelib.xml 891d9dc 
>   webapp/pom.xml ec6d836 
> 
> Diff: https://reviews.apache.org/r/25041/diff/
> 
> 
> Testing
> ---
> 
> TestHiveServer2ActionExecutor will fail against Hadoop 1 until we update to 
> Hadoop 1.2.1; though I've tested it and it should pass once we do.  I've also 
> tested the action in an actual cluster.  
> 
> 
> Thanks,
> 
> Robert Kanter
> 
>

Re: Review Request 24165: OOZIE-1954: Add a way for the MapReduce action to be configured by Java code

[shwetha . gs]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24165/#review52696
---



docs/src/site/twiki/WorkflowFunctionalSpec.twiki


Instead of depending on oozie artifact for the interface(which is difficult 
to get given that oozie artifacts are not available in any mvn repo), can the 
interface be org.apache.hadoop.util.Tool. The user provided run() can update 
the Conf. 

run() doesn't sound like setup, but serves the purpose with conf.


- shwethags


On July 31, 2014, 10:33 p.m., Robert Kanter wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/24165/
> ---
> 
> (Updated July 31, 2014, 10:33 p.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-1954
> https://issues.apache.org/jira/browse/OOZIE-1954
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> See https://issues.apache.org/jira/browse/OOZIE-1954
> 
> 
> Diffs
> -
> 
>   client/src/main/resources/oozie-workflow-0.5.xsd 6620a4e 
>   core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 
> 94b55cf 
>   core/src/test/java/org/apache/oozie/action/hadoop/MapperReducerForTest.java 
> 7fe1f42 
>   docs/src/site/twiki/WorkflowFunctionalSpec.twiki 3319bcc 
>   examples/src/main/apps/map-reduce/job-with-config-class.properties 
> PRE-CREATION 
>   examples/src/main/apps/map-reduce/job.properties 7b7a24c 
>   examples/src/main/apps/map-reduce/workflow-with-config-class.xml 
> PRE-CREATION 
>   
> examples/src/main/java/org/apache/oozie/example/SampleOozieActionConfigurator.java
>  PRE-CREATION 
>   
> sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMain.java 
> 2cb690f 
>   
> sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMapper.java
>  322f498 
>   
> sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/MapReduceMain.java
>  f417a78 
>   
> sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/OozieActionConfigurator.java
>  PRE-CREATION 
>   
> sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/OozieActionConfiguratorException.java
>  PRE-CREATION 
>   sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/PipesMain.java 
> 67a2a43 
>   
> sharelib/streaming/src/test/java/org/apache/oozie/action/hadoop/OozieActionConfiguratorForTest.java
>  PRE-CREATION 
>   
> sharelib/streaming/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java
>  e78c98a 
>   
> sharelib/streaming/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutorUberJar.java
>  bcc22f2 
> 
> Diff: https://reviews.apache.org/r/24165/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Robert Kanter
> 
>

Re: Review Request 25041: OOZIE-1457: Create a Hive Server 2 action

[shwetha . gs]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25041/#review52574
---



client/src/main/resources/hiveserver2-action-0.1.xsd


can we rename to hive2 to make it easier for people to use?



client/src/main/resources/hiveserver2-action-0.1.xsd


make it optional?



core/pom.xml


shouldn't this be provided scope?



docs/src/site/twiki/DG_HiveServer2ActionExtension.twiki






docs/src/site/twiki/DG_HiveServer2ActionExtension.twiki


Can we make password optional?


- shwethags


On Aug. 26, 2014, 12:12 a.m., Robert Kanter wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/25041/
> ---
> 
> (Updated Aug. 26, 2014, 12:12 a.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-1457
> https://issues.apache.org/jira/browse/OOZIE-1457
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> See https://issues.apache.org/jira/browse/OOZIE-1457
> 
> 
> Diffs
> -
> 
>   client/src/main/java/org/apache/oozie/cli/OozieCLI.java 33935d3 
>   client/src/main/resources/hiveserver2-action-0.1.xsd PRE-CREATION 
>   core/pom.xml 5b2eedc 
>   core/src/main/conf/oozie-site.xml 32f87f5 
>   core/src/main/java/org/apache/oozie/action/hadoop/Hive2Credentials.java 
> PRE-CREATION 
>   
> core/src/main/java/org/apache/oozie/action/hadoop/HiveServer2ActionExecutor.java
>  PRE-CREATION 
>   core/src/test/java/org/apache/oozie/test/XTestCase.java f48fff2 
>   core/src/test/java/org/apache/oozie/test/hive/AbstractHiveService.java 
> PRE-CREATION 
>   core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java PRE-CREATION 
>   docs/src/site/twiki/DG_HiveServer2ActionExtension.twiki PRE-CREATION 
>   docs/src/site/twiki/DG_UnifiedCredentialsModule.twiki 7c52970 
>   docs/src/site/twiki/index.twiki f078bf5 
>   pom.xml 835ec19 
>   sharelib/hive2/pom.xml PRE-CREATION 
>   
> sharelib/hive2/src/main/java/org/apache/oozie/action/hadoop/HiveServer2Main.java
>  PRE-CREATION 
>   
> sharelib/hive2/src/test/java/org/apache/oozie/action/hadoop/TestHiveServer2ActionExecutor.java
>  PRE-CREATION 
>   sharelib/pom.xml 1d6e4bd 
>   src/main/assemblies/sharelib.xml 891d9dc 
>   webapp/pom.xml ec6d836 
> 
> Diff: https://reviews.apache.org/r/25041/diff/
> 
> 
> Testing
> ---
> 
> TestHiveServer2ActionExecutor will fail against Hadoop 1 until we update to 
> Hadoop 1.2.1; though I've tested it and it should pass once we do.  I've also 
> tested the action in an actual cluster.  
> 
> 
> Thanks,
> 
> Robert Kanter
> 
>

Re: Review Request 24299: OOZIE-1950 Coordinator job info should support timestamp (nominal time)

[shwetha . gs]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24299/
---

(Updated Sept. 8, 2014, 6:15 a.m.)


Review request for oozie.


Changes
---

Removed dependency on external Pair. Added check on comparator for status


Bugs: OOZIE-1950
https://issues.apache.org/jira/browse/OOZIE-1950


Repository: oozie-git


Description
---

Adds filter on nominal time in get coord actions


Diffs (updated)
-

  client/src/main/java/org/apache/oozie/client/OozieClient.java 363ebd2 
  core/src/main/java/org/apache/oozie/CoordinatorEngine.java 249ae68 
  core/src/main/java/org/apache/oozie/command/coord/CoordJobXCommand.java 
4d5ba71 
  
core/src/main/java/org/apache/oozie/executor/jpa/CoordJobGetActionsSubsetJPAExecutor.java
 cdd5000 
  core/src/main/java/org/apache/oozie/util/Pair.java PRE-CREATION 
  core/src/test/java/org/apache/oozie/TestCoordinatorEngine.java 93c9235 
  core/src/test/java/org/apache/oozie/TestCoordinatorEngineSimple.java 824eb80 
  
core/src/test/java/org/apache/oozie/executor/jpa/TestCoordJobGetActionsSubsetJPAExecutor.java
 d3d29aa 
  docs/src/site/twiki/DG_CommandLineTool.twiki 4a07711 

Diff: https://reviews.apache.org/r/24299/diff/


Testing
---

UTs


Thanks,

shwethags

Re: Review Request 24299: OOZIE-1950 Coordinator job info should support timestamp (nominal time)

[shwetha . gs]

> On Aug. 29, 2014, 7:14 p.m., Purshotam Shah wrote:
> > core/src/main/java/org/apache/oozie/CoordinatorEngine.java, line 36
> > 
> >
> > [INFO] | |  +- org.apache.commons:commons-lang3:jar:3.1:provided
> > can you please check about commons-lang3, I don't think it's a part of 
> > oozie-webapp.

Didn't find anything useful in the existing libs. So, will add Pair class in 
oozie


> On Aug. 29, 2014, 7:14 p.m., Purshotam Shah wrote:
> > core/src/main/java/org/apache/oozie/CoordinatorEngine.java, line 640
> > 
> >
> > "elements must be name=value or name!=value pairs");
> > 
> > Minor.. Can you please support throwing error if status is not in 
> > format name=value or name!=value.

Will do


- shwethags


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24299/#review51896
---


On Aug. 19, 2014, 8:37 a.m., shwethags wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/24299/
> ---
> 
> (Updated Aug. 19, 2014, 8:37 a.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-1950
> https://issues.apache.org/jira/browse/OOZIE-1950
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> Adds filter on nominal time in get coord actions
> 
> 
> Diffs
> -
> 
>   client/src/main/java/org/apache/oozie/client/OozieClient.java b468186 
>   core/src/main/java/org/apache/oozie/CoordinatorEngine.java dd5c703 
>   core/src/main/java/org/apache/oozie/command/coord/CoordJobXCommand.java 
> 5eaf062 
>   
> core/src/main/java/org/apache/oozie/executor/jpa/CoordJobGetActionsSubsetJPAExecutor.java
>  a531798 
>   core/src/test/java/org/apache/oozie/TestCoordinatorEngine.java e3db675 
>   core/src/test/java/org/apache/oozie/TestCoordinatorEngineSimple.java 
> 1904519 
>   
> core/src/test/java/org/apache/oozie/executor/jpa/TestCoordJobGetActionsSubsetJPAExecutor.java
>  147bdb7 
>   docs/src/site/twiki/DG_CommandLineTool.twiki 4a07711 
> 
> Diff: https://reviews.apache.org/r/24299/diff/
> 
> 
> Testing
> ---
> 
> UTs
> 
> 
> Thanks,
> 
> shwethags
> 
>

Re: Review Request 24299: OOZIE-1950 Coordinator job info should support timestamp (nominal time)

[shwetha . gs]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24299/
---

(Updated Aug. 19, 2014, 8:37 a.m.)


Review request for oozie.


Changes
---

Changed Pair class


Bugs: OOZIE-1950
https://issues.apache.org/jira/browse/OOZIE-1950


Repository: oozie-git


Description
---

Adds filter on nominal time in get coord actions


Diffs (updated)
-

  client/src/main/java/org/apache/oozie/client/OozieClient.java b468186 
  core/src/main/java/org/apache/oozie/CoordinatorEngine.java dd5c703 
  core/src/main/java/org/apache/oozie/command/coord/CoordJobXCommand.java 
5eaf062 
  
core/src/main/java/org/apache/oozie/executor/jpa/CoordJobGetActionsSubsetJPAExecutor.java
 a531798 
  core/src/test/java/org/apache/oozie/TestCoordinatorEngine.java e3db675 
  core/src/test/java/org/apache/oozie/TestCoordinatorEngineSimple.java 1904519 
  
core/src/test/java/org/apache/oozie/executor/jpa/TestCoordJobGetActionsSubsetJPAExecutor.java
 147bdb7 
  docs/src/site/twiki/DG_CommandLineTool.twiki 4a07711 

Diff: https://reviews.apache.org/r/24299/diff/


Testing
---

UTs


Thanks,

shwethags

Re: Review Request 22615: OOZIE-1876 use pom properties rather than specific version numbers in the pom files of hbaselibs, hcataloglibs, sharelib, etc

[shwetha . gs]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/22615/
---

(Updated Aug. 19, 2014, 5:34 a.m.)


Review request for oozie.


Bugs: OOZIE-1876
https://issues.apache.org/jira/browse/OOZIE-1876


Repository: oozie-git


Description (updated)
---

1. hbase/hcatalog versions configurable
2. Include hcatalog libs in final distro


Diffs (updated)
-

  core/pom.xml 5b2eedc 
  hbaselibs/hbase-0.94/pom.xml a65904e 
  hbaselibs/pom.xml 24051c5 
  hcataloglibs/hive-hcatalog-0.13.1/pom.xml b738741 
  hcataloglibs/pom.xml a96806b 
  pom.xml c5bca2b 
  sharelib/distcp/pom.xml 04e436d 
  sharelib/hcatalog/pom.xml b934be1 
  sharelib/hive/pom.xml ea088ff 
  sharelib/pig/pom.xml ce75c68 
  sharelib/sqoop/pom.xml 09f0b66 
  sharelib/streaming/pom.xml 50d15f7 
  src/main/assemblies/distro.xml 8e2bbae 
  src/main/assemblies/hbaselib.xml 6e4db33 
  src/main/assemblies/hbaselibs.xml eb71624 
  src/main/assemblies/hcataloglib.xml 1002308 
  src/main/assemblies/hcataloglibs.xml 3d32a8f 
  tools/pom.xml a8c69cc 

Diff: https://reviews.apache.org/r/22615/diff/


Testing (updated)
---

Clean build with different hive and hbase versions


Thanks,

shwethags

Review Request 24836: OOZIE-1876 use pom properties rather than specific version numbers in the pom files of hbaselibs, hcataloglibs, sharelib, etc

[shwetha . gs]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24836/
---

Review request for oozie.


Repository: oozie-git


Description
---

Make hcatalog version configurable


Diffs
-

  core/pom.xml 5b2eedc 
  hbaselibs/hbase-0.94/pom.xml a65904e 
  hbaselibs/pom.xml 24051c5 
  hcataloglibs/hive-hcatalog-0.13.1/pom.xml b738741 
  hcataloglibs/pom.xml a96806b 
  pom.xml c5bca2b 
  sharelib/distcp/pom.xml 04e436d 
  sharelib/hcatalog/pom.xml b934be1 
  sharelib/hive/pom.xml ea088ff 
  sharelib/pig/pom.xml ce75c68 
  sharelib/sqoop/pom.xml 09f0b66 
  sharelib/streaming/pom.xml 50d15f7 
  src/main/assemblies/distro.xml 8e2bbae 
  src/main/assemblies/hbaselib.xml 6e4db33 
  src/main/assemblies/hbaselibs.xml eb71624 
  src/main/assemblies/hcataloglib.xml 1002308 
  src/main/assemblies/hcataloglibs.xml 3d32a8f 
  tools/pom.xml a8c69cc 

Diff: https://reviews.apache.org/r/24836/diff/


Testing
---

Verified oozie package


Thanks,

shwethags

Re: Review Request 23968: OOZIE-1939 Incorrect job information is set while logging

[shwetha . gs]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/23968/#review50415
---

Ship it!


Minor comment


core/src/main/java/org/apache/oozie/command/XCommand.java


not required?


- shwethags


On Aug. 11, 2014, 7:02 a.m., Azrael Park wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/23968/
> ---
> 
> (Updated Aug. 11, 2014, 7:02 a.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-1939
> https://issues.apache.org/jira/browse/OOZIE-1939
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> Because of ThreadLocal, I added setLogInfo() at XCommand.
> If the logInfo in setLogInfo() is enough, It might not need to set logInfo in 
> loadState().
> 
> 
> Diffs
> -
> 
>   core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 
> 94b55cf 
>   core/src/main/java/org/apache/oozie/command/XCommand.java 44e1133 
>   
> core/src/main/java/org/apache/oozie/command/bundle/BundleJobChangeXCommand.java
>  41ad8ae 
>   
> core/src/main/java/org/apache/oozie/command/bundle/BundleJobResumeXCommand.java
>  8ccab0e 
>   
> core/src/main/java/org/apache/oozie/command/bundle/BundleJobSuspendXCommand.java
>  65eb272 
>   core/src/main/java/org/apache/oozie/command/bundle/BundleKillXCommand.java 
> c101f45 
>   core/src/main/java/org/apache/oozie/command/bundle/BundlePauseXCommand.java 
> 0a04a02 
>   core/src/main/java/org/apache/oozie/command/bundle/BundleRerunXCommand.java 
> f5f4b3f 
>   core/src/main/java/org/apache/oozie/command/bundle/BundleStartXCommand.java 
> 74fbcab 
>   
> core/src/main/java/org/apache/oozie/command/bundle/BundleSubmitXCommand.java 
> ffb2d08 
>   
> core/src/main/java/org/apache/oozie/command/bundle/BundleUnpauseXCommand.java 
> 5d6ef65 
>   
> core/src/main/java/org/apache/oozie/command/coord/CoordActionCheckXCommand.java
>  9ac66b2 
>   
> core/src/main/java/org/apache/oozie/command/coord/CoordActionInputCheckXCommand.java
>  0a9e4fa 
>   
> core/src/main/java/org/apache/oozie/command/coord/CoordActionNotificationXCommand.java
>  562b211 
>   
> core/src/main/java/org/apache/oozie/command/coord/CoordActionReadyXCommand.java
>  ffe85f0 
>   
> core/src/main/java/org/apache/oozie/command/coord/CoordActionSkipXCommand.java
>  c774f8e 
>   
> core/src/main/java/org/apache/oozie/command/coord/CoordActionStartXCommand.java
>  1ca055d 
>   
> core/src/main/java/org/apache/oozie/command/coord/CoordActionTimeOutXCommand.java
>  5a5bf98 
>   
> core/src/main/java/org/apache/oozie/command/coord/CoordActionUpdateXCommand.java
>  6e3af70 
>   
> core/src/main/java/org/apache/oozie/command/coord/CoordActionsIgnoreXCommand.java
>  ed85a40 
>   
> core/src/main/java/org/apache/oozie/command/coord/CoordActionsKillXCommand.java
>  d2b4659 
>   core/src/main/java/org/apache/oozie/command/coord/CoordChangeXCommand.java 
> a2748c4 
>   core/src/main/java/org/apache/oozie/command/coord/CoordKillXCommand.java 
> 91e9e09 
>   
> core/src/main/java/org/apache/oozie/command/coord/CoordMaterializeTransitionXCommand.java
>  b4b2fef 
>   core/src/main/java/org/apache/oozie/command/coord/CoordPauseXCommand.java 
> cc2127a 
>   
> core/src/main/java/org/apache/oozie/command/coord/CoordPushDependencyCheckXCommand.java
>  ae71924 
>   core/src/main/java/org/apache/oozie/command/coord/CoordRerunXCommand.java 
> 735b295 
>   core/src/main/java/org/apache/oozie/command/coord/CoordResumeXCommand.java 
> 0aae1e6 
>   core/src/main/java/org/apache/oozie/command/coord/CoordSubmitXCommand.java 
> 02b30ef 
>   core/src/main/java/org/apache/oozie/command/coord/CoordSuspendXCommand.java 
> b1b2fcd 
>   core/src/main/java/org/apache/oozie/command/coord/CoordUnpauseXCommand.java 
> a177101 
>   core/src/main/java/org/apache/oozie/command/coord/CoordUpdateXCommand.java 
> d6e47da 
>   core/src/main/java/org/apache/oozie/command/wf/ActionCheckXCommand.java 
> 39c522d 
>   core/src/main/java/org/apache/oozie/command/wf/ActionEndXCommand.java 
> 42cb290 
>   core/src/main/java/org/apache/oozie/command/wf/ActionKillXCommand.java 
> 863bf7d 
>   core/src/main/java/org/apache/oozie/command/wf/ActionStartXCommand.java 
> f3a4ba9 
>   core/src/main/java/org/apache/oozie/command/wf/ActionXCommand.java 0c4c548 
>   core/src/main/java/org/apache/oozie/command/wf/CompletedActionXCommand.java 
> 41e6732 
>   core/src/main/java/org/apache/oozie/command/wf/DefinitionXCommand.java 
> 64d4822 
>   core/src/main/java/org/apache/oozie/command/wf/KillXCommand.java 37a2f8b 
>   core/src/main/java/org/apache/oozie/command/wf/NotificationXCommand.java 
> 651b5fd 
>   core/src/main/java/org/apache/oozie/command/wf/ReRunXCommand.java 5dd06ca 
>   core/src/

Re: Review Request 24299: OOZIE-1950 Coordinator job info should support timestamp (nominal time)

[shwetha . gs]

> On Aug. 12, 2014, 6:08 a.m., Rohini Palaniswamy wrote:
> > docs/src/site/twiki/DG_CommandLineTool.twiki, lines 59-60
> > 
> >
> > Revert breaking into multiple lines.

precommit build fails with longer line:(


- shwethags


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24299/#review50292
---


On Aug. 8, 2014, 9:24 a.m., shwethags wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/24299/
> ---
> 
> (Updated Aug. 8, 2014, 9:24 a.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-1950
> https://issues.apache.org/jira/browse/OOZIE-1950
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> Adds filter on nominal time in get coord actions
> 
> 
> Diffs
> -
> 
>   client/src/main/java/org/apache/oozie/client/OozieClient.java b468186 
>   core/src/main/java/org/apache/oozie/CoordinatorEngine.java dd5c703 
>   core/src/main/java/org/apache/oozie/command/coord/CoordJobXCommand.java 
> 5eaf062 
>   
> core/src/main/java/org/apache/oozie/executor/jpa/CoordJobGetActionsSubsetJPAExecutor.java
>  a531798 
>   core/src/test/java/org/apache/oozie/TestCoordinatorEngineSimple.java 
> 1904519 
>   
> core/src/test/java/org/apache/oozie/executor/jpa/TestCoordJobGetActionsSubsetJPAExecutor.java
>  147bdb7 
>   docs/src/site/twiki/DG_CommandLineTool.twiki 4a07711 
> 
> Diff: https://reviews.apache.org/r/24299/diff/
> 
> 
> Testing
> ---
> 
> UTs
> 
> 
> Thanks,
> 
> shwethags
> 
>

Re: Review Request 24299: OOZIE-1950 Coordinator job info should support timestamp (nominal time)

[shwetha . gs]

> On Aug. 8, 2014, 8:33 a.m., Purshotam Shah wrote:
> > core/src/main/java/org/apache/oozie/CoordinatorEngine.java, line 37
> > 
> >
> > Why Hbase util? This means that Oozie always need have hbase-*.jar, 
> > which might cause other conflict.
> 
> shwethags wrote:
> Pair is defined in a lot of dependent jars. So, wanted to re-use instead 
> of defining our own. If we remove hbase dependency, we can switch to others 
> or define our own. Looks ok for now
> 
> Rohini Palaniswamy wrote:
> Ah. Did not notice that earlier. This needs to be changed. We cannot 
> introduce a dependency on hbase. HBase is supposedly pluggable.

Why does oozie depend on hbase by the way?


- shwethags


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24299/#review50019
---


On Aug. 8, 2014, 9:24 a.m., shwethags wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/24299/
> ---
> 
> (Updated Aug. 8, 2014, 9:24 a.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-1950
> https://issues.apache.org/jira/browse/OOZIE-1950
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> Adds filter on nominal time in get coord actions
> 
> 
> Diffs
> -
> 
>   client/src/main/java/org/apache/oozie/client/OozieClient.java b468186 
>   core/src/main/java/org/apache/oozie/CoordinatorEngine.java dd5c703 
>   core/src/main/java/org/apache/oozie/command/coord/CoordJobXCommand.java 
> 5eaf062 
>   
> core/src/main/java/org/apache/oozie/executor/jpa/CoordJobGetActionsSubsetJPAExecutor.java
>  a531798 
>   core/src/test/java/org/apache/oozie/TestCoordinatorEngineSimple.java 
> 1904519 
>   
> core/src/test/java/org/apache/oozie/executor/jpa/TestCoordJobGetActionsSubsetJPAExecutor.java
>  147bdb7 
>   docs/src/site/twiki/DG_CommandLineTool.twiki 4a07711 
> 
> Diff: https://reviews.apache.org/r/24299/diff/
> 
> 
> Testing
> ---
> 
> UTs
> 
> 
> Thanks,
> 
> shwethags
> 
>

Re: Review Request 24299: OOZIE-1950 Coordinator job info should support timestamp (nominal time)

[shwetha . gs]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24299/
---

(Updated Aug. 8, 2014, 9:24 a.m.)


Review request for oozie.


Changes
---

comments addressed


Bugs: OOZIE-1950
https://issues.apache.org/jira/browse/OOZIE-1950


Repository: oozie-git


Description
---

Adds filter on nominal time in get coord actions


Diffs (updated)
-

  client/src/main/java/org/apache/oozie/client/OozieClient.java b468186 
  core/src/main/java/org/apache/oozie/CoordinatorEngine.java dd5c703 
  core/src/main/java/org/apache/oozie/command/coord/CoordJobXCommand.java 
5eaf062 
  
core/src/main/java/org/apache/oozie/executor/jpa/CoordJobGetActionsSubsetJPAExecutor.java
 a531798 
  core/src/test/java/org/apache/oozie/TestCoordinatorEngineSimple.java 1904519 
  
core/src/test/java/org/apache/oozie/executor/jpa/TestCoordJobGetActionsSubsetJPAExecutor.java
 147bdb7 
  docs/src/site/twiki/DG_CommandLineTool.twiki 4a07711 

Diff: https://reviews.apache.org/r/24299/diff/


Testing
---

UTs


Thanks,

shwethags

Re: Review Request 24299: OOZIE-1950 Coordinator job info should support timestamp (nominal time)

[shwetha . gs]

> On Aug. 8, 2014, 8:33 a.m., Purshotam Shah wrote:
> > core/src/main/java/org/apache/oozie/CoordinatorEngine.java, line 37
> > 
> >
> > Why Hbase util? This means that Oozie always need have hbase-*.jar, 
> > which might cause other conflict.

Pair is defined in a lot of dependent jars. So, wanted to re-use instead of 
defining our own. If we remove hbase dependency, we can switch to others or 
define our own. Looks ok for now


> On Aug. 8, 2014, 8:33 a.m., Purshotam Shah wrote:
> > core/src/main/java/org/apache/oozie/CoordinatorEngine.java, line 636
> > 
> >
> > when user pass status as 
> > status=[;status=]
> > 
> > Then it should as check for both status.
> > 
> > Eg. status=RUNNING;status=RUNNINGWITHERROR
> > 
> > Should return all status matching RUNNING or RUNNINGWITHERROR.
> > 
> > Not sure if it's happening here. 
> > 
> > From Doc.
> > "-filter  status=[;status=]* or 
> > status!=[;status!=]*
> >   All coordinator actions 
> > satisfying the status filters will be retrieved.
> >   Positive filters '=' concatenated 
> > with 'OR' and negative filters '!=' with 'AND'."
> >

Its handled correctly:
+case EQUALS:
+sb.append("IN (");
+params.putAll(appendParams(sb, filter.getValue(), pcnt));
+sb.append(")");
+break;


- shwethags


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24299/#review50019
---


On Aug. 5, 2014, 6:52 a.m., shwethags wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/24299/
> ---
> 
> (Updated Aug. 5, 2014, 6:52 a.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-1950
> https://issues.apache.org/jira/browse/OOZIE-1950
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> Adds filter on nominal time in get coord actions
> 
> 
> Diffs
> -
> 
>   client/src/main/java/org/apache/oozie/client/OozieClient.java b468186 
>   core/src/main/java/org/apache/oozie/CoordinatorEngine.java dd5c703 
>   core/src/main/java/org/apache/oozie/command/coord/CoordJobXCommand.java 
> 5eaf062 
>   
> core/src/main/java/org/apache/oozie/executor/jpa/CoordJobGetActionsSubsetJPAExecutor.java
>  a531798 
>   core/src/main/java/org/apache/oozie/servlet/V1JobServlet.java 855fabc 
>   core/src/test/java/org/apache/oozie/TestCoordinatorEngineSimple.java 
> 1904519 
>   
> core/src/test/java/org/apache/oozie/executor/jpa/TestCoordJobGetActionsSubsetJPAExecutor.java
>  147bdb7 
>   docs/src/site/twiki/DG_CommandLineTool.twiki 4a07711 
>   pom.xml be5e84a 
>   src/main/resources/checkstyle.xml b75d737 
> 
> Diff: https://reviews.apache.org/r/24299/diff/
> 
> 
> Testing
> ---
> 
> UTs
> 
> 
> Thanks,
> 
> shwethags
> 
>

Re: Review Request 23968: OOZIE-1939 Incorrect job information is set while logging

[shwetha . gs]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/23968/#review50020
---


+1 with some minor comments


core/src/main/java/org/apache/oozie/servlet/HostnameFilter.java


clear() should also resetPrefix? else, log() will use the old prefix



core/src/main/java/org/apache/oozie/util/XLog.java


JsonRestServlet has this code:
auditLog.setMsgPrefix("");

audit log logs using XLog but doesn't use XLog.Info.prefix. To support 
that, we might have to use XLog.prefix as well. 

In all XLog constructors, can we get rid of setting XLog.prefix. 
XLog.setMsgPrefix() can just set XLog.prefix without null check. XLog.log() can 
then use XLog.prefix if not null else use Info.get().getPrefix(). With this, 
since the class loggers don't set prefix, they always use 
Info.get().getPrefix(). Audit log sets XLog.prefix, so it never uses 
Info.get().getPrefix()


- shwethags


On Aug. 8, 2014, 5:31 a.m., Azrael Park wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/23968/
> ---
> 
> (Updated Aug. 8, 2014, 5:31 a.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-1939
> https://issues.apache.org/jira/browse/OOZIE-1939
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> Because of ThreadLocal, I added setLogInfo() at XCommand.
> If the logInfo in setLogInfo() is enough, It might not need to set logInfo in 
> loadState().
> 
> 
> Diffs
> -
> 
>   core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 
> 94b55cf 
>   core/src/main/java/org/apache/oozie/command/XCommand.java 44e1133 
>   
> core/src/main/java/org/apache/oozie/command/bundle/BundleJobChangeXCommand.java
>  41ad8ae 
>   
> core/src/main/java/org/apache/oozie/command/bundle/BundleJobResumeXCommand.java
>  8ccab0e 
>   
> core/src/main/java/org/apache/oozie/command/bundle/BundleJobSuspendXCommand.java
>  65eb272 
>   core/src/main/java/org/apache/oozie/command/bundle/BundleKillXCommand.java 
> c101f45 
>   core/src/main/java/org/apache/oozie/command/bundle/BundlePauseXCommand.java 
> 0a04a02 
>   core/src/main/java/org/apache/oozie/command/bundle/BundleRerunXCommand.java 
> f5f4b3f 
>   core/src/main/java/org/apache/oozie/command/bundle/BundleStartXCommand.java 
> 74fbcab 
>   
> core/src/main/java/org/apache/oozie/command/bundle/BundleSubmitXCommand.java 
> ffb2d08 
>   
> core/src/main/java/org/apache/oozie/command/bundle/BundleUnpauseXCommand.java 
> 5d6ef65 
>   
> core/src/main/java/org/apache/oozie/command/coord/CoordActionCheckXCommand.java
>  9ac66b2 
>   
> core/src/main/java/org/apache/oozie/command/coord/CoordActionInputCheckXCommand.java
>  0a9e4fa 
>   
> core/src/main/java/org/apache/oozie/command/coord/CoordActionNotificationXCommand.java
>  562b211 
>   
> core/src/main/java/org/apache/oozie/command/coord/CoordActionReadyXCommand.java
>  ffe85f0 
>   
> core/src/main/java/org/apache/oozie/command/coord/CoordActionSkipXCommand.java
>  c774f8e 
>   
> core/src/main/java/org/apache/oozie/command/coord/CoordActionStartXCommand.java
>  1ca055d 
>   
> core/src/main/java/org/apache/oozie/command/coord/CoordActionTimeOutXCommand.java
>  5a5bf98 
>   
> core/src/main/java/org/apache/oozie/command/coord/CoordActionUpdateXCommand.java
>  6e3af70 
>   
> core/src/main/java/org/apache/oozie/command/coord/CoordActionsIgnoreXCommand.java
>  ed85a40 
>   
> core/src/main/java/org/apache/oozie/command/coord/CoordActionsKillXCommand.java
>  d2b4659 
>   core/src/main/java/org/apache/oozie/command/coord/CoordChangeXCommand.java 
> a2748c4 
>   core/src/main/java/org/apache/oozie/command/coord/CoordKillXCommand.java 
> 91e9e09 
>   
> core/src/main/java/org/apache/oozie/command/coord/CoordMaterializeTransitionXCommand.java
>  b4b2fef 
>   core/src/main/java/org/apache/oozie/command/coord/CoordPauseXCommand.java 
> cc2127a 
>   
> core/src/main/java/org/apache/oozie/command/coord/CoordPushDependencyCheckXCommand.java
>  ae71924 
>   core/src/main/java/org/apache/oozie/command/coord/CoordRerunXCommand.java 
> 735b295 
>   core/src/main/java/org/apache/oozie/command/coord/CoordResumeXCommand.java 
> 0aae1e6 
>   core/src/main/java/org/apache/oozie/command/coord/CoordSubmitXCommand.java 
> 02b30ef 
>   core/src/main/java/org/apache/oozie/command/coord/CoordSuspendXCommand.java 
> b1b2fcd 
>   core/src/main/java/org/apache/oozie/command/coord/CoordUnpauseXCommand.java 
> a177101 
>   core/src/main/java/org/apache/oozie/command/coord/CoordUpdateXCommand.java 
> d6e47da 
>   core/src/main/java/org/apache/oozie/command/wf/ActionCheckXCommand.java 
> 39c522d 
>   core/src/main/ja

Re: Review Request 24299: OOZIE-1950 Coordinator job info should support timestamp (nominal time)

[shwetha . gs]

> On Aug. 7, 2014, 8:34 a.m., Rohini Palaniswamy wrote:
> > docs/src/site/twiki/DG_CommandLineTool.twiki, line 45
> > 
> >
> > Below changes can make the help a little more clear.
> > 
> >   - The comparators are not applicable for all keys. So need to clarify 
> > that =, != is applicable for status and =, !=, <, <=, >, >= applicable for 
> > nominaltime. 
> >   - Would be good to have an example.
> >
> 
> shwethags wrote:
> All comparators apply on all keys. = applies on nominal time to get 
> single action. Just like in sql
> 
> Example is in later section of filter
> 
> Rohini Palaniswamy wrote:
> For status, only = and != makes sense.

Yes, but with adding more keys in the filter, this will become complicated 
conditions. So, we can leave it as is?


> On Aug. 7, 2014, 8:34 a.m., Rohini Palaniswamy wrote:
> > src/main/resources/checkstyle.xml, line 31
> > 
> >
> > Where is this property defined? Not able to find it in code base.
> 
> shwethags wrote:
> in main pom.xml. This helps to build each module separately
> 
> Rohini Palaniswamy wrote:
> The main pom.xml does not have checkstyle.header.file defined currently. 
> I did search *.xml file in codebase.

The change in pom.xml:
src/main/resources/checkstyle-header.txt


- shwethags


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24299/#review49880
---


On Aug. 5, 2014, 6:52 a.m., shwethags wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/24299/
> ---
> 
> (Updated Aug. 5, 2014, 6:52 a.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-1950
> https://issues.apache.org/jira/browse/OOZIE-1950
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> Adds filter on nominal time in get coord actions
> 
> 
> Diffs
> -
> 
>   client/src/main/java/org/apache/oozie/client/OozieClient.java b468186 
>   core/src/main/java/org/apache/oozie/CoordinatorEngine.java dd5c703 
>   core/src/main/java/org/apache/oozie/command/coord/CoordJobXCommand.java 
> 5eaf062 
>   
> core/src/main/java/org/apache/oozie/executor/jpa/CoordJobGetActionsSubsetJPAExecutor.java
>  a531798 
>   core/src/main/java/org/apache/oozie/servlet/V1JobServlet.java 855fabc 
>   core/src/test/java/org/apache/oozie/TestCoordinatorEngineSimple.java 
> 1904519 
>   
> core/src/test/java/org/apache/oozie/executor/jpa/TestCoordJobGetActionsSubsetJPAExecutor.java
>  147bdb7 
>   docs/src/site/twiki/DG_CommandLineTool.twiki 4a07711 
>   pom.xml be5e84a 
>   src/main/resources/checkstyle.xml b75d737 
> 
> Diff: https://reviews.apache.org/r/24299/diff/
> 
> 
> Testing
> ---
> 
> UTs
> 
> 
> Thanks,
> 
> shwethags
> 
>

Re: Review Request 24299: OOZIE-1950 Coordinator job info should support timestamp (nominal time)

[shwetha . gs]

> On Aug. 7, 2014, 8:34 a.m., Rohini Palaniswamy wrote:
> > core/src/main/java/org/apache/oozie/executor/jpa/CoordJobGetActionsSubsetJPAExecutor.java,
> >  lines 126-127
> > 
> >
> > To be removed

yes, will remove


> On Aug. 7, 2014, 8:34 a.m., Rohini Palaniswamy wrote:
> > client/src/main/java/org/apache/oozie/client/OozieClient.java, line 125
> > 
> >
> > nominaltime or nominal_time - to be consistent with other filter params 
> > which don't use camel casing. I see that case does not matter later due to 
> > equalsIgnorecase check, but still just to keep naming consistent.

will rename


> On Aug. 7, 2014, 8:34 a.m., Rohini Palaniswamy wrote:
> > core/src/main/java/org/apache/oozie/servlet/V1JobServlet.java, line 827
> > 
> >
> > unintended change? Needs to be reverted.

Is this intentional of not returning any actions by default? Even some user had 
questions on why its not returning any actions and then he had to set len


> On Aug. 7, 2014, 8:34 a.m., Rohini Palaniswamy wrote:
> > core/src/test/java/org/apache/oozie/executor/jpa/TestCoordJobGetActionsSubsetJPAExecutor.java,
> >  line 89
> > 
> >
> > Please remove this line

ok


> On Aug. 7, 2014, 8:34 a.m., Rohini Palaniswamy wrote:
> > docs/src/site/twiki/DG_CommandLineTool.twiki, line 43
> > 
> >
> > default is 1000

Its 0 in V1JobServlet.java


> On Aug. 7, 2014, 8:34 a.m., Rohini Palaniswamy wrote:
> > docs/src/site/twiki/DG_CommandLineTool.twiki, line 45
> > 
> >
> > Below changes can make the help a little more clear.
> > 
> >   - The comparators are not applicable for all keys. So need to clarify 
> > that =, != is applicable for status and =, !=, <, <=, >, >= applicable for 
> > nominaltime. 
> >   - Would be good to have an example.
> >

All comparators apply on all keys. = applies on nominal time to get single 
action. Just like in sql

Example is in later section of filter


> On Aug. 7, 2014, 8:34 a.m., Rohini Palaniswamy wrote:
> > src/main/resources/checkstyle.xml, line 31
> > 
> >
> > Where is this property defined? Not able to find it in code base.

in main pom.xml. This helps to build each module separately


- shwethags


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24299/#review49880
---


On Aug. 5, 2014, 6:52 a.m., shwethags wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/24299/
> ---
> 
> (Updated Aug. 5, 2014, 6:52 a.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-1950
> https://issues.apache.org/jira/browse/OOZIE-1950
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> Adds filter on nominal time in get coord actions
> 
> 
> Diffs
> -
> 
>   client/src/main/java/org/apache/oozie/client/OozieClient.java b468186 
>   core/src/main/java/org/apache/oozie/CoordinatorEngine.java dd5c703 
>   core/src/main/java/org/apache/oozie/command/coord/CoordJobXCommand.java 
> 5eaf062 
>   
> core/src/main/java/org/apache/oozie/executor/jpa/CoordJobGetActionsSubsetJPAExecutor.java
>  a531798 
>   core/src/main/java/org/apache/oozie/servlet/V1JobServlet.java 855fabc 
>   core/src/test/java/org/apache/oozie/TestCoordinatorEngineSimple.java 
> 1904519 
>   
> core/src/test/java/org/apache/oozie/executor/jpa/TestCoordJobGetActionsSubsetJPAExecutor.java
>  147bdb7 
>   docs/src/site/twiki/DG_CommandLineTool.twiki 4a07711 
>   pom.xml be5e84a 
>   src/main/resources/checkstyle.xml b75d737 
> 
> Diff: https://reviews.apache.org/r/24299/diff/
> 
> 
> Testing
> ---
> 
> UTs
> 
> 
> Thanks,
> 
> shwethags
> 
>

Re: Review Request 23968: OOZIE-1939 Incorrect job information is set while logging

[shwetha . gs]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/23968/#review49873
---



core/src/main/java/org/apache/oozie/util/LogUtils.java


resetPrefix() on XLog doesn't work as XLog is shared across threads, so 
log() shouldn't even use XLog.prefix(with the exception for audit log which 
doesn't use thread level prefix). What you need is resetPrefix() on LogInfo 
which re-computes prefix and stores in some variable. log() should use this 
prefix


- shwethags


On Aug. 6, 2014, 6:40 a.m., Azrael Park wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/23968/
> ---
> 
> (Updated Aug. 6, 2014, 6:40 a.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-1939
> https://issues.apache.org/jira/browse/OOZIE-1939
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> Because of ThreadLocal, I added setLogInfo() at XCommand.
> If the logInfo in setLogInfo() is enough, It might not need to set logInfo in 
> loadState().
> 
> 
> Diffs
> -
> 
>   core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 
> 94b55cf 
>   core/src/main/java/org/apache/oozie/command/XCommand.java 44e1133 
>   
> core/src/main/java/org/apache/oozie/command/bundle/BundleJobChangeXCommand.java
>  41ad8ae 
>   
> core/src/main/java/org/apache/oozie/command/bundle/BundleJobResumeXCommand.java
>  8ccab0e 
>   
> core/src/main/java/org/apache/oozie/command/bundle/BundleJobSuspendXCommand.java
>  65eb272 
>   core/src/main/java/org/apache/oozie/command/bundle/BundleKillXCommand.java 
> c101f45 
>   core/src/main/java/org/apache/oozie/command/bundle/BundlePauseXCommand.java 
> 0a04a02 
>   core/src/main/java/org/apache/oozie/command/bundle/BundleRerunXCommand.java 
> f5f4b3f 
>   core/src/main/java/org/apache/oozie/command/bundle/BundleStartXCommand.java 
> 74fbcab 
>   
> core/src/main/java/org/apache/oozie/command/bundle/BundleSubmitXCommand.java 
> ffb2d08 
>   
> core/src/main/java/org/apache/oozie/command/bundle/BundleUnpauseXCommand.java 
> 5d6ef65 
>   
> core/src/main/java/org/apache/oozie/command/coord/CoordActionCheckXCommand.java
>  9ac66b2 
>   
> core/src/main/java/org/apache/oozie/command/coord/CoordActionInputCheckXCommand.java
>  0a9e4fa 
>   
> core/src/main/java/org/apache/oozie/command/coord/CoordActionNotificationXCommand.java
>  562b211 
>   
> core/src/main/java/org/apache/oozie/command/coord/CoordActionReadyXCommand.java
>  ffe85f0 
>   
> core/src/main/java/org/apache/oozie/command/coord/CoordActionSkipXCommand.java
>  c774f8e 
>   
> core/src/main/java/org/apache/oozie/command/coord/CoordActionStartXCommand.java
>  1ca055d 
>   
> core/src/main/java/org/apache/oozie/command/coord/CoordActionTimeOutXCommand.java
>  5a5bf98 
>   
> core/src/main/java/org/apache/oozie/command/coord/CoordActionUpdateXCommand.java
>  6e3af70 
>   
> core/src/main/java/org/apache/oozie/command/coord/CoordActionsIgnoreXCommand.java
>  ed85a40 
>   
> core/src/main/java/org/apache/oozie/command/coord/CoordActionsKillXCommand.java
>  d2b4659 
>   core/src/main/java/org/apache/oozie/command/coord/CoordChangeXCommand.java 
> a2748c4 
>   core/src/main/java/org/apache/oozie/command/coord/CoordKillXCommand.java 
> 91e9e09 
>   
> core/src/main/java/org/apache/oozie/command/coord/CoordMaterializeTransitionXCommand.java
>  b4b2fef 
>   core/src/main/java/org/apache/oozie/command/coord/CoordPauseXCommand.java 
> cc2127a 
>   
> core/src/main/java/org/apache/oozie/command/coord/CoordPushDependencyCheckXCommand.java
>  ae71924 
>   core/src/main/java/org/apache/oozie/command/coord/CoordRerunXCommand.java 
> 735b295 
>   core/src/main/java/org/apache/oozie/command/coord/CoordResumeXCommand.java 
> 0aae1e6 
>   core/src/main/java/org/apache/oozie/command/coord/CoordSubmitXCommand.java 
> 02b30ef 
>   core/src/main/java/org/apache/oozie/command/coord/CoordSuspendXCommand.java 
> b1b2fcd 
>   core/src/main/java/org/apache/oozie/command/coord/CoordUnpauseXCommand.java 
> a177101 
>   core/src/main/java/org/apache/oozie/command/coord/CoordUpdateXCommand.java 
> d6e47da 
>   core/src/main/java/org/apache/oozie/command/wf/ActionCheckXCommand.java 
> 39c522d 
>   core/src/main/java/org/apache/oozie/command/wf/ActionEndXCommand.java 
> 42cb290 
>   core/src/main/java/org/apache/oozie/command/wf/ActionKillXCommand.java 
> 863bf7d 
>   core/src/main/java/org/apache/oozie/command/wf/ActionStartXCommand.java 
> f3a4ba9 
>   core/src/main/java/org/apache/oozie/command/wf/ActionXCommand.java 0c4c548 
>   core/src/main/java/org/apache/oozie/command/wf/CompletedActionXCommand.java 
> 41e6732 
>   core/src/main/java/org/apache/oozie/command/wf/DefinitionXCommand.java 
> 6

Re: Review Request 23968: OOZIE-1939 Incorrect job information is set while logging

[shwetha . gs]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/23968/#review49595
---


We also need LogInfo.clearPrefix() in one of the filters for tomcat threadpool

- shwethags


On Aug. 5, 2014, 7:49 a.m., Azrael Park wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/23968/
> ---
> 
> (Updated Aug. 5, 2014, 7:49 a.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-1939
> https://issues.apache.org/jira/browse/OOZIE-1939
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> Because of ThreadLocal, I added setLogInfo() at XCommand.
> If the logInfo in setLogInfo() is enough, It might not need to set logInfo in 
> loadState().
> 
> 
> Diffs
> -
> 
>   core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 
> 94b55cf 
>   core/src/main/java/org/apache/oozie/command/XCommand.java 44e1133 
>   
> core/src/main/java/org/apache/oozie/command/bundle/BundleJobChangeXCommand.java
>  41ad8ae 
>   
> core/src/main/java/org/apache/oozie/command/bundle/BundleJobResumeXCommand.java
>  8ccab0e 
>   
> core/src/main/java/org/apache/oozie/command/bundle/BundleJobSuspendXCommand.java
>  65eb272 
>   core/src/main/java/org/apache/oozie/command/bundle/BundleKillXCommand.java 
> c101f45 
>   core/src/main/java/org/apache/oozie/command/bundle/BundlePauseXCommand.java 
> 0a04a02 
>   core/src/main/java/org/apache/oozie/command/bundle/BundleRerunXCommand.java 
> f5f4b3f 
>   core/src/main/java/org/apache/oozie/command/bundle/BundleStartXCommand.java 
> 74fbcab 
>   
> core/src/main/java/org/apache/oozie/command/bundle/BundleSubmitXCommand.java 
> ffb2d08 
>   
> core/src/main/java/org/apache/oozie/command/bundle/BundleUnpauseXCommand.java 
> 5d6ef65 
>   
> core/src/main/java/org/apache/oozie/command/coord/CoordActionCheckXCommand.java
>  9ac66b2 
>   
> core/src/main/java/org/apache/oozie/command/coord/CoordActionInputCheckXCommand.java
>  0a9e4fa 
>   
> core/src/main/java/org/apache/oozie/command/coord/CoordActionNotificationXCommand.java
>  562b211 
>   
> core/src/main/java/org/apache/oozie/command/coord/CoordActionReadyXCommand.java
>  ffe85f0 
>   
> core/src/main/java/org/apache/oozie/command/coord/CoordActionSkipXCommand.java
>  c774f8e 
>   
> core/src/main/java/org/apache/oozie/command/coord/CoordActionStartXCommand.java
>  1ca055d 
>   
> core/src/main/java/org/apache/oozie/command/coord/CoordActionTimeOutXCommand.java
>  5a5bf98 
>   
> core/src/main/java/org/apache/oozie/command/coord/CoordActionUpdateXCommand.java
>  6e3af70 
>   
> core/src/main/java/org/apache/oozie/command/coord/CoordActionsIgnoreXCommand.java
>  ed85a40 
>   
> core/src/main/java/org/apache/oozie/command/coord/CoordActionsKillXCommand.java
>  d2b4659 
>   core/src/main/java/org/apache/oozie/command/coord/CoordChangeXCommand.java 
> a2748c4 
>   core/src/main/java/org/apache/oozie/command/coord/CoordKillXCommand.java 
> 91e9e09 
>   
> core/src/main/java/org/apache/oozie/command/coord/CoordMaterializeTransitionXCommand.java
>  b4b2fef 
>   core/src/main/java/org/apache/oozie/command/coord/CoordPauseXCommand.java 
> cc2127a 
>   
> core/src/main/java/org/apache/oozie/command/coord/CoordPushDependencyCheckXCommand.java
>  ae71924 
>   core/src/main/java/org/apache/oozie/command/coord/CoordRerunXCommand.java 
> 735b295 
>   core/src/main/java/org/apache/oozie/command/coord/CoordResumeXCommand.java 
> 0aae1e6 
>   core/src/main/java/org/apache/oozie/command/coord/CoordSubmitXCommand.java 
> 02b30ef 
>   core/src/main/java/org/apache/oozie/command/coord/CoordSuspendXCommand.java 
> b1b2fcd 
>   core/src/main/java/org/apache/oozie/command/coord/CoordUnpauseXCommand.java 
> a177101 
>   core/src/main/java/org/apache/oozie/command/coord/CoordUpdateXCommand.java 
> d6e47da 
>   core/src/main/java/org/apache/oozie/command/wf/ActionCheckXCommand.java 
> 39c522d 
>   core/src/main/java/org/apache/oozie/command/wf/ActionEndXCommand.java 
> 42cb290 
>   core/src/main/java/org/apache/oozie/command/wf/ActionKillXCommand.java 
> 863bf7d 
>   core/src/main/java/org/apache/oozie/command/wf/ActionStartXCommand.java 
> f3a4ba9 
>   core/src/main/java/org/apache/oozie/command/wf/ActionXCommand.java 0c4c548 
>   core/src/main/java/org/apache/oozie/command/wf/CompletedActionXCommand.java 
> 41e6732 
>   core/src/main/java/org/apache/oozie/command/wf/DefinitionXCommand.java 
> 64d4822 
>   core/src/main/java/org/apache/oozie/command/wf/KillXCommand.java 37a2f8b 
>   core/src/main/java/org/apache/oozie/command/wf/NotificationXCommand.java 
> 651b5fd 
>   core/src/main/java/org/apache/oozie/command/wf/ReRunXCommand.java 5dd06ca 
>   core/src/main/java/org/apache/oozie/command/wf/ResumeXCommand.java 7b8ee3b 
>   core/sr

Re: Review Request 23968: OOZIE-1939 Incorrect job information is set while logging

[shwetha . gs]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/23968/#review49586
---



core/src/main/java/org/apache/oozie/service/CallableQueueService.java


We don't need to call callable.setLogInfo() as the data wouldn't be loaded 
at this point. Just XLog.Info.get().clear() will be enough



core/src/main/java/org/apache/oozie/service/CallableQueueService.java


Don't need afterExecute()



core/src/main/java/org/apache/oozie/util/XLog.java


Instead of calling createPrefix() for every log(), we can call 
createPrefix() in LogUtils.setLogInfo() which stores prefix. log() can just get 
this prefix


- shwethags


On Aug. 5, 2014, 7:49 a.m., Azrael Park wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/23968/
> ---
> 
> (Updated Aug. 5, 2014, 7:49 a.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-1939
> https://issues.apache.org/jira/browse/OOZIE-1939
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> Because of ThreadLocal, I added setLogInfo() at XCommand.
> If the logInfo in setLogInfo() is enough, It might not need to set logInfo in 
> loadState().
> 
> 
> Diffs
> -
> 
>   core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 
> 94b55cf 
>   core/src/main/java/org/apache/oozie/command/XCommand.java 44e1133 
>   
> core/src/main/java/org/apache/oozie/command/bundle/BundleJobChangeXCommand.java
>  41ad8ae 
>   
> core/src/main/java/org/apache/oozie/command/bundle/BundleJobResumeXCommand.java
>  8ccab0e 
>   
> core/src/main/java/org/apache/oozie/command/bundle/BundleJobSuspendXCommand.java
>  65eb272 
>   core/src/main/java/org/apache/oozie/command/bundle/BundleKillXCommand.java 
> c101f45 
>   core/src/main/java/org/apache/oozie/command/bundle/BundlePauseXCommand.java 
> 0a04a02 
>   core/src/main/java/org/apache/oozie/command/bundle/BundleRerunXCommand.java 
> f5f4b3f 
>   core/src/main/java/org/apache/oozie/command/bundle/BundleStartXCommand.java 
> 74fbcab 
>   
> core/src/main/java/org/apache/oozie/command/bundle/BundleSubmitXCommand.java 
> ffb2d08 
>   
> core/src/main/java/org/apache/oozie/command/bundle/BundleUnpauseXCommand.java 
> 5d6ef65 
>   
> core/src/main/java/org/apache/oozie/command/coord/CoordActionCheckXCommand.java
>  9ac66b2 
>   
> core/src/main/java/org/apache/oozie/command/coord/CoordActionInputCheckXCommand.java
>  0a9e4fa 
>   
> core/src/main/java/org/apache/oozie/command/coord/CoordActionNotificationXCommand.java
>  562b211 
>   
> core/src/main/java/org/apache/oozie/command/coord/CoordActionReadyXCommand.java
>  ffe85f0 
>   
> core/src/main/java/org/apache/oozie/command/coord/CoordActionSkipXCommand.java
>  c774f8e 
>   
> core/src/main/java/org/apache/oozie/command/coord/CoordActionStartXCommand.java
>  1ca055d 
>   
> core/src/main/java/org/apache/oozie/command/coord/CoordActionTimeOutXCommand.java
>  5a5bf98 
>   
> core/src/main/java/org/apache/oozie/command/coord/CoordActionUpdateXCommand.java
>  6e3af70 
>   
> core/src/main/java/org/apache/oozie/command/coord/CoordActionsIgnoreXCommand.java
>  ed85a40 
>   
> core/src/main/java/org/apache/oozie/command/coord/CoordActionsKillXCommand.java
>  d2b4659 
>   core/src/main/java/org/apache/oozie/command/coord/CoordChangeXCommand.java 
> a2748c4 
>   core/src/main/java/org/apache/oozie/command/coord/CoordKillXCommand.java 
> 91e9e09 
>   
> core/src/main/java/org/apache/oozie/command/coord/CoordMaterializeTransitionXCommand.java
>  b4b2fef 
>   core/src/main/java/org/apache/oozie/command/coord/CoordPauseXCommand.java 
> cc2127a 
>   
> core/src/main/java/org/apache/oozie/command/coord/CoordPushDependencyCheckXCommand.java
>  ae71924 
>   core/src/main/java/org/apache/oozie/command/coord/CoordRerunXCommand.java 
> 735b295 
>   core/src/main/java/org/apache/oozie/command/coord/CoordResumeXCommand.java 
> 0aae1e6 
>   core/src/main/java/org/apache/oozie/command/coord/CoordSubmitXCommand.java 
> 02b30ef 
>   core/src/main/java/org/apache/oozie/command/coord/CoordSuspendXCommand.java 
> b1b2fcd 
>   core/src/main/java/org/apache/oozie/command/coord/CoordUnpauseXCommand.java 
> a177101 
>   core/src/main/java/org/apache/oozie/command/coord/CoordUpdateXCommand.java 
> d6e47da 
>   core/src/main/java/org/apache/oozie/command/wf/ActionCheckXCommand.java 
> 39c522d 
>   core/src/main/java/org/apache/oozie/command/wf/ActionEndXCommand.java 
> 42cb290 
>   core/src/main/java/org/apache/oozie/command/wf/ActionKillXCommand.java 
> 863bf7d 
>   core/src/main/java/org/apache/oozie/command/wf/ActionStartXCommand.java 
>

Review Request 24299: OOZIE-1950 Coordinator job info should support timestamp (nominal time)

[shwetha . gs]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24299/
---

Review request for oozie.


Bugs: OOZIE-1950
https://issues.apache.org/jira/browse/OOZIE-1950


Repository: oozie-git


Description
---

Adds filter on nominal time in get coord actions


Diffs
-

  client/src/main/java/org/apache/oozie/client/OozieClient.java b468186 
  core/src/main/java/org/apache/oozie/CoordinatorEngine.java dd5c703 
  core/src/main/java/org/apache/oozie/command/coord/CoordJobXCommand.java 
5eaf062 
  
core/src/main/java/org/apache/oozie/executor/jpa/CoordJobGetActionsSubsetJPAExecutor.java
 a531798 
  core/src/main/java/org/apache/oozie/servlet/V1JobServlet.java 855fabc 
  core/src/test/java/org/apache/oozie/TestCoordinatorEngineSimple.java 1904519 
  
core/src/test/java/org/apache/oozie/executor/jpa/TestCoordJobGetActionsSubsetJPAExecutor.java
 147bdb7 
  docs/src/site/twiki/DG_CommandLineTool.twiki 4a07711 
  pom.xml be5e84a 
  src/main/resources/checkstyle.xml b75d737 

Diff: https://reviews.apache.org/r/24299/diff/


Testing
---

UTs


Thanks,

shwethags

Re: oozie patch

[Shwetha GS]

Manually triggered the job


On Fri, Aug 1, 2014 at 12:08 PM, Sreepathi 
wrote:

> Hello Dev's,
>
> I submitted my first patch to oozie via,
>
> https://issues.apache.org/jira/browse/OOZIE-1945
>
> and markted the jira with, patch available.  I'm not sure if it triggered
> the oozie jenkins job. Can somebody let me know, if anything else needs to
> be done ?
>
> --
> *Regards,*
> --- *Sreepathi Prasanna*
>

-- 
_
The information contained in this communication is intended solely for the 
use of the individual or entity to whom it is addressed and others 
authorized to receive it. It may contain confidential or legally privileged 
information. If you are not the intended recipient you are hereby notified 
that any disclosure, copying, distribution or taking any action in reliance 
on the contents of this information is strictly prohibited and may be 
unlawful. If you have received this communication in error, please notify 
us immediately by responding to this email and then delete it from your 
system. The firm is neither liable for the proper and complete transmission 
of the information contained in this communication nor for any delay in its 
receipt.

Re: Welcome new Oozie Committer - Shwetha GS

[Shwetha GS]

Thank you all:)


On Wed, Jul 23, 2014 at 3:36 AM, Ryota Egashira <
egash...@yahoo-inc.com.invalid> wrote:

> Congrats, Shwetha!
>
> Ryota
>
> On 7/22/14, 3:02 PM, "Alejandro Abdelnur"  wrote:
>
> >Congrats, well deserved.
> >
> >
> >On Tue, Jul 22, 2014 at 2:34 PM, Robert Kanter 
> >wrote:
> >
> >> Congrats!
> >>
> >>
> >> On Tue, Jul 22, 2014 at 1:59 PM, Purshotam Shah <
> >> purus...@yahoo-inc.com.invalid> wrote:
> >>
> >> > Congrats Shwetha!
> >> >
> >> >
> >> > Puru.
> >> >
> >> > On 7/22/14, 1:56 PM, "Mona Chitnis" 
> >> wrote:
> >> >
> >> > >Congrats Shwetha! Well deserved.
> >> > >
> >> > >‹
> >> > >Mona
> >> > >
> >> > >On 7/22/14, 8:20 AM, "Rohini Palaniswamy" 
> >> > wrote:
> >> > >
> >> > >>Hi folks,
> >> > >>
> >> > >>It is my pleasure to announce that Shwetha has become the newest
> >> > >>addition to the Oozie Committers!
> >> > >>
> >> > >>Shwetha has been involved with Oozie for more than two years now
> >> and
> >> > >>has been very actively contributing for over a year now with patches
> >> and
> >> > >>reviews. She is also Committer/PPMC of Apache Falcon which is a data
> >> > >>processing and management solution for Hadoop that uses Oozie as its
> >> > >>underlying framework for executing hadoop jobs. She comes with a
> >>wealth
> >> > >>of
> >> > >>knowledge of Oozie internals and running Oozie at scale with lot of
> >>one
> >> > >>minute coordinators.
> >> > >>
> >> > >>Please, join me in congratulating Shwetha!
> >> > >>
> >> > >>Regards,
> >> > >>Rohini
> >> > >
> >> >
> >> >
> >>
>
>

-- 
_
The information contained in this communication is intended solely for the 
use of the individual or entity to whom it is addressed and others 
authorized to receive it. It may contain confidential or legally privileged 
information. If you are not the intended recipient you are hereby notified 
that any disclosure, copying, distribution or taking any action in reliance 
on the contents of this information is strictly prohibited and may be 
unlawful. If you have received this communication in error, please notify 
us immediately by responding to this email and then delete it from your 
system. The firm is neither liable for the proper and complete transmission 
of the information contained in this communication nor for any delay in its 
receipt.

Re: Welcoming new Oozie PMC member Rohini Palaniswamy

[Shwetha GS]

Congrats Rohini

Sent from my iPhone

> On Jul 2, 2014, at 1:53 AM, Mohammad Islam  wrote:
> 
> Congrats Rohini!
> 
> 
> On Tuesday, July 1, 2014 4:50 PM, Robert Kanter  wrote:
> 
> 
> 
> Congrats!
> 
> 
> 
> On Tue, Jul 1, 2014 at 4:18 PM, Ryota Egashira <
> egash...@yahoo-inc.com.invalid> wrote:
> 
>> Same, Congrats
>> Ryota
>> 
>> On 7/1/14, 4:02 PM, "Purshotam Shah" 
>> wrote:
>> 
>>> Congarts Rohini...
>>> 
 On 7/1/14, 3:45 PM, "Alejandro Abdelnur"  wrote:
 
 Welcome Rohini, well deserved!
 
 
> On Tue, Jul 1, 2014 at 3:41 PM, Mona Chitnis  wrote:
> 
> Hello Oozie community,
> 
> 
> It's my pleasure to announce that Rohini Palaniswamy became the newest
> addition to the Oozie PMC.
> Rohini has been strongly contributing to Oozie and providing expert
> guidance.
> 
> Please join me in congratulating Rohini!
> 
> 
> 
>   --
> Mona Chitnis
> Hadoop team,
> Yahoo!
>> 

-- 
_
The information contained in this communication is intended solely for the 
use of the individual or entity to whom it is addressed and others 
authorized to receive it. It may contain confidential or legally privileged 
information. If you are not the intended recipient you are hereby notified 
that any disclosure, copying, distribution or taking any action in reliance 
on the contents of this information is strictly prohibited and may be 
unlawful. If you have received this communication in error, please notify 
us immediately by responding to this email and then delete it from your 
system. The firm is neither liable for the proper and complete transmission 
of the information contained in this communication nor for any delay in its 
receipt.

Re: Review Request 19449: OOZIE-1741: Add new coord EL function to get input partitions value string

[shwetha . gs]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/19449/#review46036
---

Ship it!


Ship It!

- shwethags


On June 17, 2014, 11:01 a.m., Satish Mittal wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/19449/
> ---
> 
> (Updated June 17, 2014, 11:01 a.m.)
> 
> 
> Review request for oozie, Ryota Egashira and Rohini Palaniswamy.
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> Currently oozie provides coord EL functions to get partition filter string 
> (where key-values are separated by AND) for various action types - 
> java/pig/hive. However this doesn't work in other cases, e.g. hive action 
> that performs export/import of hive partition. In that case, the partition 
> value is expected to be in a different format: 
> (col1="value1",col2="value2"...). 
> 
> If we use any existing EL function like coord:dataInPartitionFilter, the 
> value of partition looks like:
> (minute='00' AND month='03' AND year='2014' AND hour='09' AND day='11')
> In this case, Hive export fails with error:
> FAILED: ParseException line 1:51 mismatched input 'AND' expecting ) near 
> ''00'' in export statement
> 
> We need to add a new EL function that returns partition value in the above 
> format.
> 
> 
> Diffs
> -
> 
>   core/src/main/java/org/apache/oozie/coord/HCatELFunctions.java e5f0146 
>   core/src/main/resources/oozie-default.xml b944d3d 
>   core/src/test/java/org/apache/oozie/coord/TestHCatELFunctions.java f63b682 
>   docs/src/site/twiki/CoordinatorFunctionalSpec.twiki 961377e 
>   sharelib/hcatalog/src/main/java/org/apache/oozie/util/HCatURI.java d797f9b 
> 
> Diff: https://reviews.apache.org/r/19449/diff/
> 
> 
> Testing
> ---
> 
> - Manually verified the new EL function on my local setup.
> - Added new test cases for the same.
> 
> 
> Thanks,
> 
> Satish Mittal
> 
>

Re: Review Request 22615: OOZIE-1876 use pom properties rather than specific version numbers in the pom files of hbaselibs, hcataloglibs, sharelib, etc

[shwetha . gs]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/22615/
---

(Updated June 17, 2014, 7:12 a.m.)


Review request for oozie.


Bugs: OOZIE-1876
https://issues.apache.org/jira/browse/OOZIE-1876


Repository: oozie-git


Description (updated)
---

1. hadoop/hbase/hcatalog versions configurable
2. Include hadooplibs in final distro
3. Renamed oozie-default.xml in final distro

Usage:
mvn clean package [-Dhadoop.version=] - hadoop-1 build
mvn clean package -P hadoop-2 [-Dhadoop.version=] - hadoop-2 build
mvn clean package -P hadoop-3 [-Dhadoop.version=] - hadoop-3 build
mvn clean package -P hadoop-23 [-Dhadoop.version=] - hadoop-0.23 build


Diffs
-

  core/pom.xml c935dd7 
  distro/pom.xml 25370c0 
  examples/pom.xml b70ffec 
  hadooplibs/hadoop-0.23/pom.xml f01975e 
  hadooplibs/hadoop-1/pom.xml 30a5996 
  hadooplibs/hadoop-2/pom.xml a02422a 
  hadooplibs/hadoop-3/pom.xml b63bbe4 
  hadooplibs/hadoop-distcp-0.23/pom.xml 36414ba 
  hadooplibs/hadoop-distcp-1/pom.xml fa21703 
  hadooplibs/hadoop-distcp-2/pom.xml d5d37d6 
  hadooplibs/hadoop-distcp-3/pom.xml 491407e 
  hadooplibs/hadoop-distcp/pom.xml PRE-CREATION 
  hadooplibs/hadoop-libs/pom.xml PRE-CREATION 
  hadooplibs/hadoop-test-0.23/pom.xml b0bef89 
  hadooplibs/hadoop-test-1/pom.xml 3d4b409 
  hadooplibs/hadoop-test-2/pom.xml 944dba4 
  hadooplibs/hadoop-test-3/pom.xml e1a3a93 
  hadooplibs/hadoop-utils-0.23/pom.xml 13e736f 
  
hadooplibs/hadoop-utils-0.23/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java
 07a3c29 
  hadooplibs/hadoop-utils-1/pom.xml 0d10e7a 
  
hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java
 07a3c29 
  hadooplibs/hadoop-utils-2/pom.xml ce6d531 
  
hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java
 07a3c29 
  hadooplibs/hadoop-utils-3/pom.xml 07afb2d 
  
hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java
 db827e5 
  hadooplibs/hadoop-utils/pom.xml PRE-CREATION 
  
hadooplibs/hadoop-utils/src/versioned-src/hadoop-1/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java
 PRE-CREATION 
  
hadooplibs/hadoop-utils/src/versioned-src/hadoop-2/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java
 PRE-CREATION 
  
hadooplibs/hadoop-utils/src/versioned-src/hadoop-23/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java
 PRE-CREATION 
  
hadooplibs/hadoop-utils/src/versioned-src/hadoop-3/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java
 PRE-CREATION 
  hadooplibs/pom.xml 788788c 
  hbaselibs/hbase-0.94/pom.xml 6b2526c 
  hbaselibs/pom.xml 8369121 
  hcataloglibs/hcatalog-0.5/pom.xml 457874d 
  hcataloglibs/hcatalog-0.6/pom.xml 98ae13e 
  hcataloglibs/pom.xml cde7bb9 
  minitest/pom.xml 0a6030f 
  pom.xml b5e0e4e 
  sharelib/distcp/pom.xml 486162a 
  sharelib/hcatalog/pom.xml 4efa5e0 
  sharelib/hive/pom.xml 5ca5cb8 
  sharelib/oozie/pom.xml 22a01b6 
  sharelib/pig/pom.xml 6f99523 
  sharelib/sqoop/pom.xml 8925376 
  sharelib/streaming/pom.xml aceab41 
  src/main/assemblies/distro.xml 8e2bbae 
  src/main/assemblies/hadooplib.xml b32ca82 
  src/main/assemblies/hadooplibs.xml 9884efa 
  src/main/assemblies/hbaselib.xml 6e4db33 
  src/main/assemblies/hbaselibs.xml eb71624 
  src/main/assemblies/hcataloglib.xml 1002308 
  src/main/assemblies/hcataloglibs.xml d00d34b 
  tools/pom.xml 487783d 
  webapp/pom.xml 93cfcef 

Diff: https://reviews.apache.org/r/22615/diff/


Testing
---

Clean build with different hadoop profiles, verified distro that it contains 
the same files as before


Thanks,

shwethags

Re: Review Request 22615: OOZIE-1876 use pom properties rather than specific version numbers in the pom files of hbaselibs, hcataloglibs, sharelib, etc

[shwetha . gs]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/22615/
---

(Updated June 17, 2014, 7:11 a.m.)


Review request for oozie.


Changes
---

Added new property for hadoop-auth version


Bugs: OOZIE-1876
https://issues.apache.org/jira/browse/OOZIE-1876


Repository: oozie-git


Description
---

1. hadoop/hbase/hcatalog versions configurable
2. Include hadooplibs in final distro
3. Renamed oozie-default.xml in final distro


Diffs (updated)
-

  core/pom.xml c935dd7 
  distro/pom.xml 25370c0 
  examples/pom.xml b70ffec 
  hadooplibs/hadoop-0.23/pom.xml f01975e 
  hadooplibs/hadoop-1/pom.xml 30a5996 
  hadooplibs/hadoop-2/pom.xml a02422a 
  hadooplibs/hadoop-3/pom.xml b63bbe4 
  hadooplibs/hadoop-distcp-0.23/pom.xml 36414ba 
  hadooplibs/hadoop-distcp-1/pom.xml fa21703 
  hadooplibs/hadoop-distcp-2/pom.xml d5d37d6 
  hadooplibs/hadoop-distcp-3/pom.xml 491407e 
  hadooplibs/hadoop-distcp/pom.xml PRE-CREATION 
  hadooplibs/hadoop-libs/pom.xml PRE-CREATION 
  hadooplibs/hadoop-test-0.23/pom.xml b0bef89 
  hadooplibs/hadoop-test-1/pom.xml 3d4b409 
  hadooplibs/hadoop-test-2/pom.xml 944dba4 
  hadooplibs/hadoop-test-3/pom.xml e1a3a93 
  hadooplibs/hadoop-utils-0.23/pom.xml 13e736f 
  
hadooplibs/hadoop-utils-0.23/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java
 07a3c29 
  hadooplibs/hadoop-utils-1/pom.xml 0d10e7a 
  
hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java
 07a3c29 
  hadooplibs/hadoop-utils-2/pom.xml ce6d531 
  
hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java
 07a3c29 
  hadooplibs/hadoop-utils-3/pom.xml 07afb2d 
  
hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java
 db827e5 
  hadooplibs/hadoop-utils/pom.xml PRE-CREATION 
  
hadooplibs/hadoop-utils/src/versioned-src/hadoop-1/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java
 PRE-CREATION 
  
hadooplibs/hadoop-utils/src/versioned-src/hadoop-2/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java
 PRE-CREATION 
  
hadooplibs/hadoop-utils/src/versioned-src/hadoop-23/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java
 PRE-CREATION 
  
hadooplibs/hadoop-utils/src/versioned-src/hadoop-3/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java
 PRE-CREATION 
  hadooplibs/pom.xml 788788c 
  hbaselibs/hbase-0.94/pom.xml 6b2526c 
  hbaselibs/pom.xml 8369121 
  hcataloglibs/hcatalog-0.5/pom.xml 457874d 
  hcataloglibs/hcatalog-0.6/pom.xml 98ae13e 
  hcataloglibs/pom.xml cde7bb9 
  minitest/pom.xml 0a6030f 
  pom.xml b5e0e4e 
  sharelib/distcp/pom.xml 486162a 
  sharelib/hcatalog/pom.xml 4efa5e0 
  sharelib/hive/pom.xml 5ca5cb8 
  sharelib/oozie/pom.xml 22a01b6 
  sharelib/pig/pom.xml 6f99523 
  sharelib/sqoop/pom.xml 8925376 
  sharelib/streaming/pom.xml aceab41 
  src/main/assemblies/distro.xml 8e2bbae 
  src/main/assemblies/hadooplib.xml b32ca82 
  src/main/assemblies/hadooplibs.xml 9884efa 
  src/main/assemblies/hbaselib.xml 6e4db33 
  src/main/assemblies/hbaselibs.xml eb71624 
  src/main/assemblies/hcataloglib.xml 1002308 
  src/main/assemblies/hcataloglibs.xml d00d34b 
  tools/pom.xml 487783d 
  webapp/pom.xml 93cfcef 

Diff: https://reviews.apache.org/r/22615/diff/


Testing
---

Clean build with different hadoop profiles, verified distro that it contains 
the same files as before


Thanks,

shwethags

Re: Review Request 19449: OOZIE-1741: Add new coord EL function to get input partitions value string

[shwetha . gs]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/19449/#review45914
---



core/src/main/java/org/apache/oozie/coord/HCatELFunctions.java


Can you specify dataInName in the exceptions. When there are multiple data 
ins for a coord, its difficult to figure out for which data in the error is for



core/src/main/java/org/apache/oozie/coord/HCatELFunctions.java


There should be validation on type in the beginning of the function



core/src/test/java/org/apache/oozie/coord/TestHCatELFunctions.java


why do you need assertEquals here?



core/src/test/java/org/apache/oozie/coord/TestHCatELFunctions.java


why do you need assertEquals here?



core/src/test/java/org/apache/oozie/coord/TestHCatELFunctions.java


Can you add another test with different type (not hive-export)



docs/src/site/twiki/CoordinatorFunctionalSpec.twiki


Uses dataInPartitions for output



docs/src/site/twiki/CoordinatorFunctionalSpec.twiki


extra)



sharelib/hcatalog/src/main/java/org/apache/oozie/util/HCatURI.java


throw unhandled type


- shwethags


On June 16, 2014, 1:09 p.m., Satish Mittal wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/19449/
> ---
> 
> (Updated June 16, 2014, 1:09 p.m.)
> 
> 
> Review request for oozie, Ryota Egashira and Rohini Palaniswamy.
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> Currently oozie provides coord EL functions to get partition filter string 
> (where key-values are separated by AND) for various action types - 
> java/pig/hive. However this doesn't work in other cases, e.g. hive action 
> that performs export/import of hive partition. In that case, the partition 
> value is expected to be in a different format: 
> (col1="value1",col2="value2"...). 
> 
> If we use any existing EL function like coord:dataInPartitionFilter, the 
> value of partition looks like:
> (minute='00' AND month='03' AND year='2014' AND hour='09' AND day='11')
> In this case, Hive export fails with error:
> FAILED: ParseException line 1:51 mismatched input 'AND' expecting ) near 
> ''00'' in export statement
> 
> We need to add a new EL function that returns partition value in the above 
> format.
> 
> 
> Diffs
> -
> 
>   core/src/main/java/org/apache/oozie/coord/HCatELFunctions.java e5f0146 
>   core/src/main/resources/oozie-default.xml c9d9591 
>   core/src/test/java/org/apache/oozie/coord/TestHCatELFunctions.java f63b682 
>   docs/src/site/twiki/CoordinatorFunctionalSpec.twiki 11018b0 
>   sharelib/hcatalog/src/main/java/org/apache/oozie/util/HCatURI.java d797f9b 
> 
> Diff: https://reviews.apache.org/r/19449/diff/
> 
> 
> Testing
> ---
> 
> - Manually verified the new EL function on my local setup.
> - Added new test cases for the same.
> 
> 
> Thanks,
> 
> Satish Mittal
> 
>

Re: Review Request 19449: OOZIE-1741: Add new coord EL function to get input partitions value string

[shwetha . gs]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/19449/#review45732
---



core/src/main/java/org/apache/oozie/coord/HCatELFunctions.java


pass e as argument. Not in message (so that stactrace is available)


- shwethags


On June 13, 2014, 5:40 p.m., Satish Mittal wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/19449/
> ---
> 
> (Updated June 13, 2014, 5:40 p.m.)
> 
> 
> Review request for oozie, Ryota Egashira and Rohini Palaniswamy.
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> Currently oozie provides coord EL functions to get partition filter string 
> (where key-values are separated by AND) for various action types - 
> java/pig/hive. However this doesn't work in other cases, e.g. hive action 
> that performs export/import of hive partition. In that case, the partition 
> value is expected to be in a different format: 
> (col1="value1",col2="value2"...). 
> 
> If we use any existing EL function like coord:dataInPartitionFilter, the 
> value of partition looks like:
> (minute='00' AND month='03' AND year='2014' AND hour='09' AND day='11')
> In this case, Hive export fails with error:
> FAILED: ParseException line 1:51 mismatched input 'AND' expecting ) near 
> ''00'' in export statement
> 
> We need to add a new EL function that returns partition value in the above 
> format.
> 
> 
> Diffs
> -
> 
>   core/src/main/java/org/apache/oozie/coord/HCatELFunctions.java e5f0146 
>   core/src/main/resources/oozie-default.xml 63a91fa 
>   core/src/test/java/org/apache/oozie/coord/TestHCatELFunctions.java f63b682 
>   docs/src/site/twiki/CoordinatorFunctionalSpec.twiki 11018b0 
>   sharelib/hcatalog/src/main/java/org/apache/oozie/util/HCatURI.java d797f9b 
> 
> Diff: https://reviews.apache.org/r/19449/diff/
> 
> 
> Testing
> ---
> 
> - Manually verified the new EL function on my local setup.
> - Added new test cases for the same.
> 
> 
> Thanks,
> 
> Satish Mittal
> 
>

Review Request 22615: OOZIE-1876 use pom properties rather than specific version numbers in the pom files of hbaselibs, hcataloglibs, sharelib, etc

[shwetha . gs]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/22615/
---

Review request for oozie.


Bugs: OOZIE-1876
https://issues.apache.org/jira/browse/OOZIE-1876


Repository: oozie-git


Description
---

1. hadoop/hbase/hcatalog versions configurable
2. Include hadooplibs in final distro
3. Renamed oozie-default.xml in final distro


Diffs
-

  core/pom.xml c935dd7 
  distro/pom.xml 25370c0 
  examples/pom.xml b70ffec 
  hadooplibs/hadoop-0.23/pom.xml f01975e 
  hadooplibs/hadoop-1/pom.xml 30a5996 
  hadooplibs/hadoop-2/pom.xml a02422a 
  hadooplibs/hadoop-3/pom.xml b63bbe4 
  hadooplibs/hadoop-distcp-0.23/pom.xml 36414ba 
  hadooplibs/hadoop-distcp-1/pom.xml fa21703 
  hadooplibs/hadoop-distcp-2/pom.xml d5d37d6 
  hadooplibs/hadoop-distcp-3/pom.xml 491407e 
  hadooplibs/hadoop-distcp/pom.xml PRE-CREATION 
  hadooplibs/hadoop-libs/pom.xml PRE-CREATION 
  hadooplibs/hadoop-test-0.23/pom.xml b0bef89 
  hadooplibs/hadoop-test-1/pom.xml 3d4b409 
  hadooplibs/hadoop-test-2/pom.xml 944dba4 
  hadooplibs/hadoop-test-3/pom.xml e1a3a93 
  hadooplibs/hadoop-utils-0.23/pom.xml 13e736f 
  
hadooplibs/hadoop-utils-0.23/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java
 07a3c29 
  hadooplibs/hadoop-utils-1/pom.xml 0d10e7a 
  
hadooplibs/hadoop-utils-1/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java
 07a3c29 
  hadooplibs/hadoop-utils-2/pom.xml ce6d531 
  
hadooplibs/hadoop-utils-2/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java
 07a3c29 
  hadooplibs/hadoop-utils-3/pom.xml 07afb2d 
  
hadooplibs/hadoop-utils-3/src/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java
 db827e5 
  hadooplibs/hadoop-utils/pom.xml PRE-CREATION 
  
hadooplibs/hadoop-utils/src/versioned-src/hadoop-1/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java
 PRE-CREATION 
  
hadooplibs/hadoop-utils/src/versioned-src/hadoop-2/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java
 PRE-CREATION 
  
hadooplibs/hadoop-utils/src/versioned-src/hadoop-23/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java
 PRE-CREATION 
  
hadooplibs/hadoop-utils/src/versioned-src/hadoop-3/main/java/org/apache/oozie/action/hadoop/LauncherMainHadoopUtils.java
 PRE-CREATION 
  hadooplibs/pom.xml 788788c 
  hbaselibs/hbase-0.94/pom.xml 6b2526c 
  hbaselibs/pom.xml 8369121 
  hcataloglibs/hcatalog-0.5/pom.xml 457874d 
  hcataloglibs/hcatalog-0.6/pom.xml 98ae13e 
  hcataloglibs/pom.xml cde7bb9 
  minitest/pom.xml 0a6030f 
  pom.xml b5e0e4e 
  sharelib/distcp/pom.xml 486162a 
  sharelib/hcatalog/pom.xml 4efa5e0 
  sharelib/hive/pom.xml 5ca5cb8 
  sharelib/oozie/pom.xml 22a01b6 
  sharelib/pig/pom.xml 6f99523 
  sharelib/sqoop/pom.xml 8925376 
  sharelib/streaming/pom.xml aceab41 
  src/main/assemblies/distro.xml 8e2bbae 
  src/main/assemblies/hadooplib.xml b32ca82 
  src/main/assemblies/hadooplibs.xml 9884efa 
  src/main/assemblies/hbaselib.xml 6e4db33 
  src/main/assemblies/hbaselibs.xml eb71624 
  src/main/assemblies/hcataloglib.xml 1002308 
  src/main/assemblies/hcataloglibs.xml d00d34b 
  tools/pom.xml 487783d 
  webapp/pom.xml 93cfcef 

Diff: https://reviews.apache.org/r/22615/diff/


Testing
---

Clean build with different hadoop profiles, verified distro that it contains 
the same files as before


Thanks,

shwethags

Re: Review Request 22491: OOZIE-1846 Convert CoordActionMaterializeCommand to an XCommand and remove Command

[shwetha . gs]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/22491/#review45719
---

Ship it!


Ship It!

- shwethags


On June 16, 2014, 5:48 a.m., Azrael Park wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/22491/
> ---
> 
> (Updated June 16, 2014, 5:48 a.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-1846
> https://issues.apache.org/jira/browse/OOZIE-1846
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> Use CoordActionMaterializeCommand for dryrun and remove Command.
> 
> Change surefire version to 2.12.2 for single method test.
> 
> 
> Diffs
> -
> 
>   core/src/main/java/org/apache/oozie/command/Command.java d3f1011 
>   core/src/main/java/org/apache/oozie/command/CommandException.java ce5ef54 
>   
> core/src/main/java/org/apache/oozie/command/coord/CoordActionMaterializeCommand.java
>  6962fb2 
>   
> core/src/main/java/org/apache/oozie/command/coord/CoordMaterializeTransitionXCommand.java
>  515e247 
>   core/src/main/java/org/apache/oozie/command/coord/CoordSubmitXCommand.java 
> 91fb5da 
>   core/src/main/java/org/apache/oozie/command/coord/CoordinatorCommand.java 
> c70e171 
>   
> core/src/main/java/org/apache/oozie/service/CoordMaterializeTriggerService.java
>  d2b5a6c 
>   core/src/test/java/org/apache/oozie/command/TestCommand.java 60363bf 
>   
> core/src/test/java/org/apache/oozie/command/coord/TestCoordActionMaterializeCommand.java
>  6278515 
>   
> core/src/test/java/org/apache/oozie/command/coord/TestCoordELExtensions.java 
> 7f7a387 
>   
> core/src/test/java/org/apache/oozie/command/coord/TestCoordMaterializeTransitionXCommand.java
>  5b22abc 
>   pom.xml b5e0e4e 
> 
> Diff: https://reviews.apache.org/r/22491/diff/
> 
> 
> Testing
> ---
> 
> Unit test done.
> Manual test done.
> 
> 
> Thanks,
> 
> Azrael Park
> 
>

Re: Review Request 22491: OOZIE-1846 Convert CoordActionMaterializeCommand to an XCommand and remove Command

[shwetha . gs]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/22491/#review45603
---



core/src/test/java/org/apache/oozie/command/coord/TestCoordMaterializeTransitionXCommand.java


For dryRun, the coord is not in DB. So, can you change 
addRecordToCoordJobTable to createCoordJob. 

Also checkCoordActions is not required. 



- shwethags


On June 13, 2014, 7:05 a.m., Azrael Park wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/22491/
> ---
> 
> (Updated June 13, 2014, 7:05 a.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-1846
> https://issues.apache.org/jira/browse/OOZIE-1846
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> Use CoordActionMaterializeCommand for dryrun and remove Command.
> 
> Change surefire version to 2.12.2 for single method test.
> 
> 
> Diffs
> -
> 
>   core/src/main/java/org/apache/oozie/command/Command.java d3f1011 
>   core/src/main/java/org/apache/oozie/command/CommandException.java ce5ef54 
>   
> core/src/main/java/org/apache/oozie/command/coord/CoordActionMaterializeCommand.java
>  6962fb2 
>   
> core/src/main/java/org/apache/oozie/command/coord/CoordMaterializeTransitionXCommand.java
>  515e247 
>   core/src/main/java/org/apache/oozie/command/coord/CoordSubmitXCommand.java 
> 91fb5da 
>   core/src/main/java/org/apache/oozie/command/coord/CoordinatorCommand.java 
> c70e171 
>   core/src/test/java/org/apache/oozie/command/TestCommand.java 60363bf 
>   
> core/src/test/java/org/apache/oozie/command/coord/TestCoordActionMaterializeCommand.java
>  6278515 
>   
> core/src/test/java/org/apache/oozie/command/coord/TestCoordELExtensions.java 
> 7f7a387 
>   
> core/src/test/java/org/apache/oozie/command/coord/TestCoordMaterializeTransitionXCommand.java
>  5b22abc 
>   pom.xml b5e0e4e 
> 
> Diff: https://reviews.apache.org/r/22491/diff/
> 
> 
> Testing
> ---
> 
> Unit test done.
> Manual test done.
> 
> 
> Thanks,
> 
> Azrael Park
> 
>

Re: Review Request 22491: OOZIE-1846 Convert CoordActionMaterializeCommand to an XCommand and remove Command

[shwetha . gs]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/22491/#review45575
---


Looks good otherwise


core/src/main/java/org/apache/oozie/command/coord/CoordMaterializeTransitionXCommand.java


You can remove even CoordinatorCommand



core/src/main/java/org/apache/oozie/command/coord/CoordSubmitXCommand.java


A little refactoring which is better:
Define constructor CoordMaterializeTransitionXCommand(coordJob, startTime, 
endTime) and call materializeActions(true) directly


- shwethags


On June 12, 2014, 2:10 a.m., Azrael Park wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/22491/
> ---
> 
> (Updated June 12, 2014, 2:10 a.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-1846
> https://issues.apache.org/jira/browse/OOZIE-1846
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> Use CoordActionMaterializeCommand for dryrun and remove Command.
> 
> Change surefire version to 2.12.2 for single method test.
> 
> 
> Diffs
> -
> 
>   core/src/main/java/org/apache/oozie/command/Command.java d3f1011 
>   core/src/main/java/org/apache/oozie/command/CommandException.java ce5ef54 
>   
> core/src/main/java/org/apache/oozie/command/coord/CoordActionMaterializeCommand.java
>  6962fb2 
>   
> core/src/main/java/org/apache/oozie/command/coord/CoordMaterializeTransitionXCommand.java
>  515e247 
>   core/src/main/java/org/apache/oozie/command/coord/CoordSubmitXCommand.java 
> 91fb5da 
>   core/src/main/java/org/apache/oozie/command/coord/CoordinatorCommand.java 
> c70e171 
>   core/src/test/java/org/apache/oozie/command/TestCommand.java 60363bf 
>   
> core/src/test/java/org/apache/oozie/command/coord/TestCoordActionMaterializeCommand.java
>  6278515 
>   
> core/src/test/java/org/apache/oozie/command/coord/TestCoordELExtensions.java 
> 7f7a387 
>   
> core/src/test/java/org/apache/oozie/command/coord/TestCoordMaterializeTransitionXCommand.java
>  5b22abc 
>   pom.xml b5e0e4e 
> 
> Diff: https://reviews.apache.org/r/22491/diff/
> 
> 
> Testing
> ---
> 
> Unit test done.
> Manual test done.
> 
> 
> Thanks,
> 
> Azrael Park
> 
>

Re: Review Request 20696: OOZIE-1807 Make bundle change command synchronous

[shwetha . gs]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/20696/#review45063
---



core/src/main/java/org/apache/oozie/command/bundle/BundleJobChangeXCommand.java


Increasing the end time of killed coord is allowed with coord change API. 
So, why not allow bundle change as well if coord is killed


- shwethags


On April 25, 2014, 1:41 a.m., Purshotam Shah wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/20696/
> ---
> 
> (Updated April 25, 2014, 1:41 a.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-1807
> https://issues.apache.org/jira/browse/OOZIE-1807
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> OOZIE-1807 Make bundle change command synchronous
> 
> 
> Diffs
> -
> 
>   client/src/main/java/org/apache/oozie/cli/OozieCLI.java f26e33b 
>   client/src/main/java/org/apache/oozie/client/OozieClient.java e5afdc7 
>   client/src/main/java/org/apache/oozie/client/rest/JsonTags.java c791474 
>   core/src/main/java/org/apache/oozie/BaseEngine.java 135a0a3 
>   core/src/main/java/org/apache/oozie/BundleEngine.java 5bf1538 
>   core/src/main/java/org/apache/oozie/CoordinatorEngine.java 81d98bf 
>   core/src/main/java/org/apache/oozie/DagEngine.java cad5ddd 
>   core/src/main/java/org/apache/oozie/ErrorCode.java 8b573ce 
>   
> core/src/main/java/org/apache/oozie/command/bundle/BundleJobChangeXCommand.java
>  cd2c844 
>   core/src/main/java/org/apache/oozie/command/coord/CoordChangeXCommand.java 
> fb31e9a 
>   core/src/main/java/org/apache/oozie/servlet/BaseJobServlet.java 8941a02 
>   core/src/main/java/org/apache/oozie/servlet/V0JobServlet.java 487a371 
>   core/src/main/java/org/apache/oozie/servlet/V1JobServlet.java 5b65791 
>   
> core/src/test/java/org/apache/oozie/command/bundle/TestBundleChangeXCommand.java
>  6560db6 
>   
> core/src/test/java/org/apache/oozie/command/coord/TestCoordChangeXCommand.java
>  327ec90 
>   
> core/src/test/java/org/apache/oozie/servlet/MockCoordinatorEngineService.java 
> 9892d4b 
> 
> Diff: https://reviews.apache.org/r/20696/diff/
> 
> 
> Testing
> ---
> 
> UTC and manual testing.
> 
> purushah$ 
> /Users/purushah/views/sysc_command/oozie/distro/target/oozie-4.1.0-SNAPSHOT-distro/oozie-4.1.0-SNAPSHOT/bin/oozie
>  job -change 002-140424165425407-oozie-puru-B  -value 
> endtime=2013-01-01T00:00Z -oozie http://localhost:11000/oozie
> Change command for coord job : 004-140424165425407-oozie-puru-C failed. 
> Reason : Coord is in killed state
> Change command for coord job : 003-140424165425407-oozie-puru-C failed. 
> Reason : Coord is in killed state
> 
> purushah$ 
> 
> 
> Thanks,
> 
> Purshotam Shah
> 
>

Re: Review Request 20553: OOZIE-1243: libtools dir should not include hadoop JARs

[shwetha . gs]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/20553/#review41590
---

Ship it!


Ship It!

- shwethags


On April 28, 2014, 12:12 p.m., Satish Mittal wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/20553/
> ---
> 
> (Updated April 28, 2014, 12:12 p.m.)
> 
> 
> Review request for oozie and Alejandro Abdelnur.
> 
> 
> Bugs: OOZIE-1243
> https://issues.apache.org/jira/browse/OOZIE-1243
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> All the setup tools should consume the hadoop JARs from the libext/ 
> directory, having them in the libtools/ dir make things fail when using 
> different hadoop versions other than the one used to compile Oozie.
> 
> 
> Diffs
> -
> 
>   src/main/assemblies/tools.xml 9b65773 
>   tools/pom.xml 1245e47 
> 
> Diff: https://reviews.apache.org/r/20553/diff/
> 
> 
> Testing
> ---
> 
> - Prepared distro and ran the following command: 'oozie-setup.sh sharelib 
> create'. Started oozie successfully.
> 
> 
> Thanks,
> 
> Satish Mittal
> 
>

Re: Review Request 20553: OOZIE-1243: libtools dir should not include hadoop JARs

[shwetha . gs]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/20553/#review41589
---



src/main/assemblies/tools.xml


can you add a comment on why its runtime. Its not intuitive. Looks good 
otherwise


- shwethags


On April 28, 2014, 12:12 p.m., Satish Mittal wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/20553/
> ---
> 
> (Updated April 28, 2014, 12:12 p.m.)
> 
> 
> Review request for oozie and Alejandro Abdelnur.
> 
> 
> Bugs: OOZIE-1243
> https://issues.apache.org/jira/browse/OOZIE-1243
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> All the setup tools should consume the hadoop JARs from the libext/ 
> directory, having them in the libtools/ dir make things fail when using 
> different hadoop versions other than the one used to compile Oozie.
> 
> 
> Diffs
> -
> 
>   src/main/assemblies/tools.xml 9b65773 
>   tools/pom.xml 1245e47 
> 
> Diff: https://reviews.apache.org/r/20553/diff/
> 
> 
> Testing
> ---
> 
> - Prepared distro and ran the following command: 'oozie-setup.sh sharelib 
> create'. Started oozie successfully.
> 
> 
> Thanks,
> 
> Satish Mittal
> 
>

Re: Review Request 18762: OOZIE-1709 CoordELFunctions.getCurrentInstance() is expensive

[shwetha . gs]

> On April 10, 2014, 1:50 a.m., Purshotam Shah wrote:
> > core/src/main/java/org/apache/oozie/coord/CoordELFunctions.java, line 1363
> > 
> >
> > Why are u diving and multiplying and why if(instanceCount[0] > 2)

case MINUTE:
instanceCount[0] = (int) ((effectiveTime.getTime() - 
datasetInitialInstance.getTime()) / MINUTE_MSEC);
If the frequency is 5 minutely, this just computes the difference in minutes. 
So, the actual instance count is (instanceCount[0]/freq). For calendar 
addition, we again need to multiply by freq as the addition is at minute level. 
Makes sense?


- shwethags


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/18762/#review39973
---


On April 21, 2014, 7:02 a.m., shwethags wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/18762/
> ---
> 
> (Updated April 21, 2014, 7:02 a.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-1709
> https://issues.apache.org/jira/browse/OOZIE-1709
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> CoordELFunctions.getCurrentInstance() has this code:
> while (current.compareTo(calEffectiveTime) <= 0) {
> current = (Calendar) origCurrent.clone();
> instanceCount[0]++;
> current.add(dsTimeUnit.getCalendarUnit(), instanceCount[0] * 
> dsFreq);
> }
> 
> For coords with smaller frequency and start time in very past, this is very 
> expensive. On prod, we have seen materialisation of each instance taking few 
> mins sometimes for coords with 1 min frequency
> 
> 
> Diffs
> -
> 
>   core/src/main/java/org/apache/oozie/coord/CoordELFunctions.java d73bc7d 
>   core/src/test/java/org/apache/oozie/coord/TestCoordELFunctions.java be35ce4 
> 
> Diff: https://reviews.apache.org/r/18762/diff/
> 
> 
> Testing
> ---
> 
> UT
> 
> 
> Thanks,
> 
> shwethags
> 
>

Re: Review Request 18762: OOZIE-1709 CoordELFunctions.getCurrentInstance() is expensive

[shwetha . gs]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/18762/
---

(Updated April 21, 2014, 7:02 a.m.)


Review request for oozie.


Changes
---

Removed the old function


Bugs: OOZIE-1709
https://issues.apache.org/jira/browse/OOZIE-1709


Repository: oozie-git


Description
---

CoordELFunctions.getCurrentInstance() has this code:
while (current.compareTo(calEffectiveTime) <= 0) {
current = (Calendar) origCurrent.clone();
instanceCount[0]++;
current.add(dsTimeUnit.getCalendarUnit(), instanceCount[0] * 
dsFreq);
}

For coords with smaller frequency and start time in very past, this is very 
expensive. On prod, we have seen materialisation of each instance taking few 
mins sometimes for coords with 1 min frequency


Diffs (updated)
-

  core/src/main/java/org/apache/oozie/coord/CoordELFunctions.java d73bc7d 
  core/src/test/java/org/apache/oozie/coord/TestCoordELFunctions.java be35ce4 

Diff: https://reviews.apache.org/r/18762/diff/


Testing
---

UT


Thanks,

shwethags

Re: Review Request 18762: OOZIE-1709 CoordELFunctions.getCurrentInstance() is expensive

[shwetha . gs]

> On April 10, 2014, 1:50 a.m., Purshotam Shah wrote:
> > core/src/main/java/org/apache/oozie/coord/CoordELFunctions.java, line 1367
> > 
> >
> > Can you explain, why this is needed. 
> > 
> > Can't u get the correct offset by (effectiveTime.getTime() - 
> > datasetInitialInstance.getTime())/unit?

The code assumes 30 days per month, 365 days a year which is not right. Hence 
the count is offset by 2 and use for loop to get the correct offset


- shwethags


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/18762/#review39973
---


On April 8, 2014, 5:17 p.m., shwethags wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/18762/
> ---
> 
> (Updated April 8, 2014, 5:17 p.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-1709
> https://issues.apache.org/jira/browse/OOZIE-1709
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> CoordELFunctions.getCurrentInstance() has this code:
> while (current.compareTo(calEffectiveTime) <= 0) {
> current = (Calendar) origCurrent.clone();
> instanceCount[0]++;
> current.add(dsTimeUnit.getCalendarUnit(), instanceCount[0] * 
> dsFreq);
> }
> 
> For coords with smaller frequency and start time in very past, this is very 
> expensive. On prod, we have seen materialisation of each instance taking few 
> mins sometimes for coords with 1 min frequency
> 
> 
> Diffs
> -
> 
>   core/src/main/java/org/apache/oozie/coord/CoordELFunctions.java d73bc7d 
>   core/src/test/java/org/apache/oozie/coord/TestCoordELFunctions.java be35ce4 
> 
> Diff: https://reviews.apache.org/r/18762/diff/
> 
> 
> Testing
> ---
> 
> UT
> 
> 
> Thanks,
> 
> shwethags
> 
>

Re: Review Request 18762: OOZIE-1709 CoordELFunctions.getCurrentInstance() is expensive

[shwetha . gs]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/18762/
---

(Updated April 8, 2014, 5:17 p.m.)


Review request for oozie.


Changes
---

Changed code to avoid multiple logging


Bugs: OOZIE-1709
https://issues.apache.org/jira/browse/OOZIE-1709


Repository: oozie-git


Description
---

CoordELFunctions.getCurrentInstance() has this code:
while (current.compareTo(calEffectiveTime) <= 0) {
current = (Calendar) origCurrent.clone();
instanceCount[0]++;
current.add(dsTimeUnit.getCalendarUnit(), instanceCount[0] * 
dsFreq);
}

For coords with smaller frequency and start time in very past, this is very 
expensive. On prod, we have seen materialisation of each instance taking few 
mins sometimes for coords with 1 min frequency


Diffs (updated)
-

  core/src/main/java/org/apache/oozie/coord/CoordELFunctions.java d73bc7d 
  core/src/test/java/org/apache/oozie/coord/TestCoordELFunctions.java be35ce4 

Diff: https://reviews.apache.org/r/18762/diff/


Testing
---

UT


Thanks,

shwethags

Re: Review Request 18762: OOZIE-1709 CoordELFunctions.getCurrentInstance() is expensive

[shwetha . gs]

> On April 8, 2014, 4:26 p.m., Purshotam Shah wrote:
> > core/src/main/java/org/apache/oozie/coord/CoordELFunctions.java, line 961
> > 
> >
> > We are logging too many statement here, it might be better if we can 
> > log it only once.
> > 
> > something like this.
> > 
> > boolean shouldlog = true;
> > if (nominalInstanceCal.compareTo(initInstance) < 0) {
> > if (shouldlog) {
> > LOG.warn();
> > shouldlog = false;
> > }
> >

Makes sense. Will do


- shwethags


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/18762/#review39790
---


On April 8, 2014, 1:03 a.m., shwethags wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/18762/
> ---
> 
> (Updated April 8, 2014, 1:03 a.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-1709
> https://issues.apache.org/jira/browse/OOZIE-1709
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> CoordELFunctions.getCurrentInstance() has this code:
> while (current.compareTo(calEffectiveTime) <= 0) {
> current = (Calendar) origCurrent.clone();
> instanceCount[0]++;
> current.add(dsTimeUnit.getCalendarUnit(), instanceCount[0] * 
> dsFreq);
> }
> 
> For coords with smaller frequency and start time in very past, this is very 
> expensive. On prod, we have seen materialisation of each instance taking few 
> mins sometimes for coords with 1 min frequency
> 
> 
> Diffs
> -
> 
>   core/src/main/java/org/apache/oozie/coord/CoordELFunctions.java d73bc7d 
>   core/src/test/java/org/apache/oozie/coord/TestCoordELFunctions.java be35ce4 
> 
> Diff: https://reviews.apache.org/r/18762/diff/
> 
> 
> Testing
> ---
> 
> UT
> 
> 
> Thanks,
> 
> shwethags
> 
>

Re: Review Request 18762: OOZIE-1709 CoordELFunctions.getCurrentInstance() is expensive

[shwetha . gs]

> On April 8, 2014, 12:44 a.m., Purshotam Shah wrote:
> > core/src/main/java/org/apache/oozie/coord/CoordELFunctions.java, line 1314
> > 
> >
> > instanceCount, is not this same as coord.lastActionNumber?
> > 
> > At least in case of nominal time, this should same as 
> > coord.lastActionNumber. We can use that to optimize query.
> 
> shwethags wrote:
> instanceCount is the data instance count with respect to the coord 
> nominal time. coord.lastActionNumber is the last materialised action. They 
> are not related and can't be re-used
> 
> Purshotam Shah wrote:
> I missed the part that coord.lastActionNumber is offset from start time,  
> where as instanceCount is offset from initial instance.
> 
> We are still looping to get instanceCount, if we can get get rid of that 
> then we will get much better performance. For now we are good.

get instance count is the major bottle neck and I have changed that too. Check 
getCurrentInstance()


- shwethags


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/18762/#review39621
---


On April 8, 2014, 1:03 a.m., shwethags wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/18762/
> ---
> 
> (Updated April 8, 2014, 1:03 a.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-1709
> https://issues.apache.org/jira/browse/OOZIE-1709
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> CoordELFunctions.getCurrentInstance() has this code:
> while (current.compareTo(calEffectiveTime) <= 0) {
> current = (Calendar) origCurrent.clone();
> instanceCount[0]++;
> current.add(dsTimeUnit.getCalendarUnit(), instanceCount[0] * 
> dsFreq);
> }
> 
> For coords with smaller frequency and start time in very past, this is very 
> expensive. On prod, we have seen materialisation of each instance taking few 
> mins sometimes for coords with 1 min frequency
> 
> 
> Diffs
> -
> 
>   core/src/main/java/org/apache/oozie/coord/CoordELFunctions.java d73bc7d 
>   core/src/test/java/org/apache/oozie/coord/TestCoordELFunctions.java be35ce4 
> 
> Diff: https://reviews.apache.org/r/18762/diff/
> 
> 
> Testing
> ---
> 
> UT
> 
> 
> Thanks,
> 
> shwethags
> 
>

Re: Review Request 18762: OOZIE-1709 CoordELFunctions.getCurrentInstance() is expensive

[shwetha . gs]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/18762/
---

(Updated April 8, 2014, 1:03 a.m.)


Review request for oozie.


Changes
---

Updated log message


Bugs: OOZIE-1709
https://issues.apache.org/jira/browse/OOZIE-1709


Repository: oozie-git


Description
---

CoordELFunctions.getCurrentInstance() has this code:
while (current.compareTo(calEffectiveTime) <= 0) {
current = (Calendar) origCurrent.clone();
instanceCount[0]++;
current.add(dsTimeUnit.getCalendarUnit(), instanceCount[0] * 
dsFreq);
}

For coords with smaller frequency and start time in very past, this is very 
expensive. On prod, we have seen materialisation of each instance taking few 
mins sometimes for coords with 1 min frequency


Diffs (updated)
-

  core/src/main/java/org/apache/oozie/coord/CoordELFunctions.java d73bc7d 
  core/src/test/java/org/apache/oozie/coord/TestCoordELFunctions.java be35ce4 

Diff: https://reviews.apache.org/r/18762/diff/


Testing
---

UT


Thanks,

shwethags

Re: Review Request 18762: OOZIE-1709 CoordELFunctions.getCurrentInstance() is expensive

[shwetha . gs]

> On April 8, 2014, 12:44 a.m., Purshotam Shah wrote:
> > core/src/main/java/org/apache/oozie/coord/CoordELFunctions.java, line 960
> > 
> >
> > Can we put a check, we are logging same message repetitively..

will update the message


> On April 8, 2014, 12:44 a.m., Purshotam Shah wrote:
> > core/src/main/java/org/apache/oozie/coord/CoordELFunctions.java, line 1314
> > 
> >
> > instanceCount, is not this same as coord.lastActionNumber?
> > 
> > At least in case of nominal time, this should same as 
> > coord.lastActionNumber. We can use that to optimize query.

instanceCount is the data instance count with respect to the coord nominal 
time. coord.lastActionNumber is the last materialised action. They are not 
related and can't be re-used


> On April 8, 2014, 12:44 a.m., Purshotam Shah wrote:
> > core/src/main/java/org/apache/oozie/coord/CoordELFunctions.java, line 1339
> > 
> >
> > I haven't tested this. Will this work for different timezone?

yes, it should work with different timezones and DST as Calendar is used.

There are tests in TestCoordELFunctions which test with different timezones and 
DST. I haven't changed the tests and all tests pass. So, the patch shouldn't 
add any regression


- shwethags


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/18762/#review39621
---


On March 26, 2014, 10:09 a.m., shwethags wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/18762/
> ---
> 
> (Updated March 26, 2014, 10:09 a.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-1709
> https://issues.apache.org/jira/browse/OOZIE-1709
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> CoordELFunctions.getCurrentInstance() has this code:
> while (current.compareTo(calEffectiveTime) <= 0) {
> current = (Calendar) origCurrent.clone();
> instanceCount[0]++;
> current.add(dsTimeUnit.getCalendarUnit(), instanceCount[0] * 
> dsFreq);
> }
> 
> For coords with smaller frequency and start time in very past, this is very 
> expensive. On prod, we have seen materialisation of each instance taking few 
> mins sometimes for coords with 1 min frequency
> 
> 
> Diffs
> -
> 
>   core/src/main/java/org/apache/oozie/coord/CoordELFunctions.java d73bc7d 
>   core/src/test/java/org/apache/oozie/coord/TestCoordELFunctions.java 34a428f 
> 
> Diff: https://reviews.apache.org/r/18762/diff/
> 
> 
> Testing
> ---
> 
> UT
> 
> 
> Thanks,
> 
> shwethags
> 
>

Re: Review Request 18762: OOZIE-1709 CoordELFunctions.getCurrentInstance() is expensive

[shwetha . gs]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/18762/
---

(Updated March 26, 2014, 10:09 a.m.)


Review request for oozie.


Changes
---

Incorporated review comments. Added test for currentrange


Bugs: OOZIE-1709
https://issues.apache.org/jira/browse/OOZIE-1709


Repository: oozie-git


Description
---

CoordELFunctions.getCurrentInstance() has this code:
while (current.compareTo(calEffectiveTime) <= 0) {
current = (Calendar) origCurrent.clone();
instanceCount[0]++;
current.add(dsTimeUnit.getCalendarUnit(), instanceCount[0] * 
dsFreq);
}

For coords with smaller frequency and start time in very past, this is very 
expensive. On prod, we have seen materialisation of each instance taking few 
mins sometimes for coords with 1 min frequency


Diffs (updated)
-

  core/src/main/java/org/apache/oozie/coord/CoordELFunctions.java d73bc7d 
  core/src/test/java/org/apache/oozie/coord/TestCoordELFunctions.java 34a428f 

Diff: https://reviews.apache.org/r/18762/diff/


Testing
---

UT


Thanks,

shwethags

Re: Review Request 18762: OOZIE-1709 CoordELFunctions.getCurrentInstance() is expensive

[shwetha . gs]

> On March 5, 2014, 6:03 p.m., Purshotam Shah wrote:
> > core/src/main/java/org/apache/oozie/coord/CoordELFunctions.java, line 962
> > 
> >
> > I see some issue with this. Earlier we start from end and break the for 
> > loop if nominalInstanceCal.compareTo(initInstance).
> > There are still some instance being return, but not in this case.
> > 
> > Eg 
> >
> > 
> > ${coord:current(-200)}
> > ${coord:current(0)}
> > 
> > 
> > 
> > Where coord:current(-10) < initInstance.
> > 
> > Earlier it will return -9 to 0, but now none.
> > 
> > 
> >
> 
> shwethags wrote:
> Actually, instead of returning partial instances and user wondering what 
> went wrong with this coord action, we should add this check at coord submit.  
> Coord submit should fail if the instances span before/after dataset time 
> interval
> 
> Purshotam Shah wrote:
> No. User cloud be using this. User might want to process full data + 
> increment data every time.
> 
> They will set start-instance as coord:current(-100) (-very high 
> number), and end-instance as current(0), so that they can process full feed + 
> incremental feed.
> 
> This cloud be one way to process all data from start date to current(0). 
>

Better thing to do in this case will be to expose an EL function that returns 
the first instance. Till that feature is available, we should support the 
usecase that you mentioned. I will update tha patch accordingly.


> On March 5, 2014, 6:03 p.m., Purshotam Shah wrote:
> > core/src/main/java/org/apache/oozie/coord/CoordELFunctions.java, line 961
> > 
> >
> > This can be moved outside from for loop. since you are starting from 
> > start and it's increasing.
> 
> shwethags wrote:
> I will add check in coord submit. So, this check won't be required anymore

This check can't be moved outside for loop for cases when start count is -ve. 
This can evaluate to an instance before the dataset start


- shwethags


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/18762/#review36237
---


On March 5, 2014, 5:28 a.m., shwethags wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/18762/
> ---
> 
> (Updated March 5, 2014, 5:28 a.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-1709
> https://issues.apache.org/jira/browse/OOZIE-1709
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> CoordELFunctions.getCurrentInstance() has this code:
> while (current.compareTo(calEffectiveTime) <= 0) {
> current = (Calendar) origCurrent.clone();
> instanceCount[0]++;
> current.add(dsTimeUnit.getCalendarUnit(), instanceCount[0] * 
> dsFreq);
> }
> 
> For coords with smaller frequency and start time in very past, this is very 
> expensive. On prod, we have seen materialisation of each instance taking few 
> mins sometimes for coords with 1 min frequency
> 
> 
> Diffs
> -
> 
>   core/src/main/java/org/apache/oozie/coord/CoordELFunctions.java d73bc7d 
>   core/src/test/java/org/apache/oozie/coord/TestCoordELFunctions.java 34a428f 
> 
> Diff: https://reviews.apache.org/r/18762/diff/
> 
> 
> Testing
> ---
> 
> UT
> 
> 
> Thanks,
> 
> shwethags
> 
>

Re: Review Request 18762: OOZIE-1709 CoordELFunctions.getCurrentInstance() is expensive

[shwetha . gs]

> On March 5, 2014, 6:03 p.m., Purshotam Shah wrote:
> > core/src/main/java/org/apache/oozie/coord/CoordELFunctions.java, line 962
> > 
> >
> > I see some issue with this. Earlier we start from end and break the for 
> > loop if nominalInstanceCal.compareTo(initInstance).
> > There are still some instance being return, but not in this case.
> > 
> > Eg 
> >
> > 
> > ${coord:current(-200)}
> > ${coord:current(0)}
> > 
> > 
> > 
> > Where coord:current(-10) < initInstance.
> > 
> > Earlier it will return -9 to 0, but now none.
> > 
> > 
> >

Actually, instead of returning partial instances and user wondering what went 
wrong with this coord action, we should add this check at coord submit.  Coord 
submit should fail if the instances span before/after dataset time interval


> On March 5, 2014, 6:03 p.m., Purshotam Shah wrote:
> > core/src/main/java/org/apache/oozie/coord/CoordELFunctions.java, line 961
> > 
> >
> > This can be moved outside from for loop. since you are starting from 
> > start and it's increasing.

I will add check in coord submit. So, this check won't be required anymore


- shwethags


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/18762/#review36237
---


On March 5, 2014, 5:28 a.m., shwethags wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/18762/
> ---
> 
> (Updated March 5, 2014, 5:28 a.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-1709
> https://issues.apache.org/jira/browse/OOZIE-1709
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> CoordELFunctions.getCurrentInstance() has this code:
> while (current.compareTo(calEffectiveTime) <= 0) {
> current = (Calendar) origCurrent.clone();
> instanceCount[0]++;
> current.add(dsTimeUnit.getCalendarUnit(), instanceCount[0] * 
> dsFreq);
> }
> 
> For coords with smaller frequency and start time in very past, this is very 
> expensive. On prod, we have seen materialisation of each instance taking few 
> mins sometimes for coords with 1 min frequency
> 
> 
> Diffs
> -
> 
>   core/src/main/java/org/apache/oozie/coord/CoordELFunctions.java d73bc7d 
>   core/src/test/java/org/apache/oozie/coord/TestCoordELFunctions.java 34a428f 
> 
> Diff: https://reviews.apache.org/r/18762/diff/
> 
> 
> Testing
> ---
> 
> UT
> 
> 
> Thanks,
> 
> shwethags
> 
>

Re: oozie coordinator job frequency DST transition

[Shwetha GS]

Calendar.add() takes care of DST transition


On Thu, Mar 6, 2014 at 4:04 AM, bowen zhang  wrote:

> Isn't it ideal that the job is still running at 8am after March 9th? Oozie
> doc says oozie handles DST transition for timezones. I don't see here how
> oozie is really handling any DST transition.
> Bowen
>
>
>
>
>
> On Wednesday, March 5, 2014 12:04 PM, Rohini Palaniswamy <
> rohini.adi...@gmail.com> wrote:
>
> I believe so. Even we have seen this behaviour. Our users usually have the
>  timezone in UTC and only  in different
> timezones.
>
> Regards,
> Rohini
>
>
>
> On Tue, Mar 4, 2014 at 5:12 PM, bowen zhang 
> wrote:
>
> > Hi guys,
> > I have a coordinator job frequency DST transition question. If I schedule
> > a job to run at 8am everyday with frequency=1440 minutes and
> > timezone="America/Los Angeles", then the job will start running at 9am
> > after March 9th, 2014.
> > If you look at the line
> > "start.add(freqTU.getCalendarUnit(), lastActionNumber *
> > Integer.parseInt(coordJob.getFrequency()));" in
> > CoordMaterializeTransitionXCommand in materializeActions() function, this
> > line doesn't seem to take care of the DST transition. Is this the desired
> > behavior for oozie?
> > Bowen
> >
>

-- 
_
The information contained in this communication is intended solely for the 
use of the individual or entity to whom it is addressed and others 
authorized to receive it. It may contain confidential or legally privileged 
information. If you are not the intended recipient you are hereby notified 
that any disclosure, copying, distribution or taking any action in reliance 
on the contents of this information is strictly prohibited and may be 
unlawful. If you have received this communication in error, please notify 
us immediately by responding to this email and then delete it from your 
system. The firm is neither liable for the proper and complete transmission 
of the information contained in this communication nor for any delay in its 
receipt.

Review Request 18762: OOZIE-1709 CoordELFunctions.getCurrentInstance() is expensive

[shwetha . gs]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/18762/
---

Review request for oozie.


Bugs: OOZIE-1709
https://issues.apache.org/jira/browse/OOZIE-1709


Repository: oozie-git


Description
---

CoordELFunctions.getCurrentInstance() has this code:
while (current.compareTo(calEffectiveTime) <= 0) {
current = (Calendar) origCurrent.clone();
instanceCount[0]++;
current.add(dsTimeUnit.getCalendarUnit(), instanceCount[0] * 
dsFreq);
}

For coords with smaller frequency and start time in very past, this is very 
expensive. On prod, we have seen materialisation of each instance taking few 
mins sometimes for coords with 1 min frequency


Diffs
-

  core/src/main/java/org/apache/oozie/coord/CoordELFunctions.java d73bc7d 
  core/src/test/java/org/apache/oozie/coord/TestCoordELFunctions.java 34a428f 

Diff: https://reviews.apache.org/r/18762/diff/


Testing
---

UT


Thanks,

shwethags

Review please

[Shwetha GS]

Can someone review this please:
https://issues.apache.org/jira/browse/OOZIE-1709

Thanks,
Shwetha

-- 
_
The information contained in this communication is intended solely for the 
use of the individual or entity to whom it is addressed and others 
authorized to receive it. It may contain confidential or legally privileged 
information. If you are not the intended recipient you are hereby notified 
that any disclosure, copying, distribution or taking any action in reliance 
on the contents of this information is strictly prohibited and may be 
unlawful. If you have received this communication in error, please notify 
us immediately by responding to this email and then delete it from your 
system. The firm is neither liable for the proper and complete transmission 
of the information contained in this communication nor for any delay in its 
receipt.

Re: Obscure problem with SchedulerService that causes tasks to not be rescheduled

[Shwetha GS]

We had the same issue. We found that because of an edge case, the callable
command(CoordMaterializeTransitionXCommand in this case which does the
actual materialisation) is added to the unique callable(in
CallableQueueService), but is never added to the queue. Hence the
materialisation doesn't happen. Since CoordMaterializeTransitionXCommand is
already in unique callable, further recovery
from CoordMaterializeTriggerService doesn't enqueue any
more CoordMaterializeTransitionXCommand. The issue is being fixed as part
of https://issues.apache.org/jira/browse/OOZIE-1699. This issue can happen
with any command(not just materialisation).

To verify if this is indeed the issue,
1. CoordMaterializeTriggerService should pick up this coord every 5
mins(materialisation interval)
2. CoordMaterializeTransitionXCommand never runs
3. oozie admin -queuedump lists that coord for mater in unique callable




On Sat, Feb 22, 2014 at 5:29 AM, Robert Kanter  wrote:

> Sorry for the long message, but its hard to explain the issue succinctly.
>
> We recently saw a weird issue where Oozie suddenly stopped materializing
> actions for a Coordinator job.  All we saw in the logs was this message
> 2014-01-31 02:10:03,378 WARN org.apache.oozie.service.JPAService: USER[-]
> GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] JPAExecutor
> [CoordJobsToBeMaterializedJPAExecutor] ended with an active transaction,
> rolling back
>
> After a bunch of digging, I figured out what was happening:
>
> The CoordMaterializeTriggerService is responsible for materializing Coord
> actions via a Runnable that gets scheduled every 5min via the
> SchedulerService.  The SchedulerService is backed by a
> ScheduledThreadPoolExecutor.  The Runnable does a bunch of things, but the
> important thing is that it executes CoordJobsToBeMaterializedJPAExecutor
> via the JPAService.  In the JPAService#execute(...) method, this happens:
> try {
> if (em.getTransaction().isActive()) {
> LOG.warn("JPAExecutor [{0}] ended with an active transaction,
> rolling back", executor.getName());
> *em.getTransaction().rollback();*
> }
> }
> catch (Exception ex) {
> LOG.warn("Could not check/rollback transaction after JPAExecutor
> [{0}], {1}", executor.getName(), ex
> .getMessage(), ex);
> }
> I've determined that the bold line (the rollback) can throw an Error (as in
> the Throwable, not a synonym for Exception).  For obvious reasons, we're
> not catching this Error anywhere.  This causes the Thread to exit; however,
> we don't "lose" any threads because the ScheduledThreadPoolExecutor will
> simply start a new one.  However, the task that the thread was executed is
> dropped from the schedule.  According to the documentation [1] it says "If
> any execution of the task encounters an exception, subsequent executions
> are suppressed."  I did some testing, and that's not quite true.
>  Exceptions don't cause subsequent executions to be suppressed, but Errors
> do.
>
> I tried catching all Errors to figure out what was causing this and
> reproduced the problem (or at least a similar problem by killing the MySQL
> database) and got a stack trace like this (that I've shorted for brevity):
> java.lang.ExceptionInInitializerError
> at
>
> org.apache.openjpa.lib.util.ConcreteClassGenerator.newInstance(ConcreteClassGenerator.java:132)
> at
>
> org.apache.openjpa.lib.jdbc.ConfiguringConnectionDecorator.decorate(ConfiguringConnectionDecorator.java:110)
> at
>
> org.apache.openjpa.lib.jdbc.DecoratingDataSource.decorate(DecoratingDataSource.java:118)
> at
>
> org.apache.openjpa.lib.jdbc.DecoratingDataSource.getConnection(DecoratingDataSource.java:107)
> ...
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.GeneratedConstructorAccessor26.newInstance(Unknown
> Source)
> at
>
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
> at
>
> org.apache.openjpa.lib.util.ConcreteClassGenerator.newInstance(ConcreteClassGenerator.java:130)
> ... 28 more
> Caused by: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException:
> Communications link failure
> As we can see here [2], ConcreteClassGenerator.newInstance throws an
> ExceptionInInitializerError when java.lang.reflect.Constructor.newInstance
> throws any Exception; in this case, an InvocationTargetException, which it
> throws if the underlying constructor in the reflection throws an Exception
> [3].  And the underlying constructor is throwing a CommunicationsException
> from the MySQL JDBC driver.
>
> Long story short, in some cases scheduled tasks can throw an Error instead
> of an Exception when there's a database-related problem.  And this causes
> them to not be rescheduled.  It is also possible for this to happen to
> other tasks being executed via the SchedulerService besides the
> C

pending field for bundle action

[Shwetha GS]

Hi,

How is pending field used for bundle actions?

I was trying to debug why bundle was running even after it was killed and
didn't understand how is pending used. The bundle was set to RUNNING from
KILLED in StatusTransitService and bundle action was pending and KILLED.

Bundle action pending is incremented in bundle commands and decremented
in BundleStatusUpdateXCommand. But BundleStatusUpdateXCommand just updates
bundle action status because of changes in coord status. So, how is pending
useful for this? What am I missing?

Thanks,
Shwetha

-- 
_
The information contained in this communication is intended solely for the 
use of the individual or entity to whom it is addressed and others 
authorized to receive it. It may contain confidential or legally privileged 
information. If you are not the intended recipient you are hereby notified 
that any disclosure, copying, distribution or taking any action in reliance 
on the contents of this information is strictly prohibited and may be 
unlawful. If you have received this communication in error, please notify 
us immediately by responding to this email and then delete it from your 
system. The firm is neither liable for the proper and complete transmission 
of the information contained in this communication nor for any delay in its 
receipt.

Review for patch available

[Shwetha GS]

Can someone review these please?
https://issues.apache.org/jira/browse/OOZIE-1622
https://issues.apache.org/jira/browse/OOZIE-1663
https://issues.apache.org/jira/browse/OOZIE-1664

Thanks,
Shwetha

-- 
_
The information contained in this communication is intended solely for the 
use of the individual or entity to whom it is addressed and others 
authorized to receive it. It may contain confidential or legally privileged 
information. If you are not the intended recipient you are hereby notified 
that any disclosure, copying, distribution or taking any action in reliance 
on the contents of this information is strictly prohibited and may be 
unlawful. If you have received this communication in error, please notify 
us immediately by responding to this email and then delete it from your 
system. The firm is neither liable for the proper and complete transmission 
of the information contained in this communication nor for any delay in its 
receipt.

Bundle Conf with 1 coord created 2-3 coords

[Shwetha GS]

Oozie Dev,

We had a bundle which had just 1 coord in the conf. But oozie created 2/3
coords for the same bundle id. Looks like a critical bug.

I have added details and pointed out possible code issue in
https://issues.apache.org/jira/browse/OOZIE-1622. Can someone check if its
valid

Thanks,
Shwetha

-- 
_
The information contained in this communication is intended solely for the 
use of the individual or entity to whom it is addressed and others 
authorized to receive it. It may contain confidential or legally privileged 
information. If you are not the intended recipient you are hereby notified 
that any disclosure, copying, distribution or taking any action in reliance 
on the contents of this information is strictly prohibited and may be 
unlawful. If you have received this communication in error, please notify 
us immediately by responding to this email and then delete it from your 
system. The firm is neither liable for the proper and complete transmission 
of the information contained in this communication nor for any delay in its 
receipt.

Re: Review Patches

[Shwetha GS]

Thank you


On Tue, Oct 29, 2013 at 9:43 PM, Virag Kothari  wrote:

> Reviewing..
>
> Thanks,
> Virag
>
> On 10/28/13 10:52 PM, "Shwetha GS"  wrote:
>
> >Can somebody review these please:
> >https://issues.apache.org/jira/browse/OOZIE-1562
> >https://issues.apache.org/jira/browse/OOZIE-1314
> >
> >Thanks,
> >Shwetha
> >
> >--
> >_
> >The information contained in this communication is intended solely for
> >the
> >use of the individual or entity to whom it is addressed and others
> >authorized to receive it. It may contain confidential or legally
> >privileged
> >information. If you are not the intended recipient you are hereby
> >notified
> >that any disclosure, copying, distribution or taking any action in
> >reliance
> >on the contents of this information is strictly prohibited and may be
> >unlawful. If you have received this communication in error, please notify
> >us immediately by responding to this email and then delete it from your
> >system. The firm is neither liable for the proper and complete
> >transmission
> >of the information contained in this communication nor for any delay in
> >its
> >receipt.
>
>

-- 
_
The information contained in this communication is intended solely for the 
use of the individual or entity to whom it is addressed and others 
authorized to receive it. It may contain confidential or legally privileged 
information. If you are not the intended recipient you are hereby notified 
that any disclosure, copying, distribution or taking any action in reliance 
on the contents of this information is strictly prohibited and may be 
unlawful. If you have received this communication in error, please notify 
us immediately by responding to this email and then delete it from your 
system. The firm is neither liable for the proper and complete transmission 
of the information contained in this communication nor for any delay in its 
receipt.

Review Patches

[Shwetha GS]

Can somebody review these please:
https://issues.apache.org/jira/browse/OOZIE-1562
https://issues.apache.org/jira/browse/OOZIE-1314

Thanks,
Shwetha

-- 
_
The information contained in this communication is intended solely for the 
use of the individual or entity to whom it is addressed and others 
authorized to receive it. It may contain confidential or legally privileged 
information. If you are not the intended recipient you are hereby notified 
that any disclosure, copying, distribution or taking any action in reliance 
on the contents of this information is strictly prohibited and may be 
unlawful. If you have received this communication in error, please notify 
us immediately by responding to this email and then delete it from your 
system. The firm is neither liable for the proper and complete transmission 
of the information contained in this communication nor for any delay in its 
receipt.

Re: Branching for Oozie 4.0

[Shwetha GS]

https://issues.apache.org/jira/browse/OOZIE-1465 has a small patch to make
some constants public. These are very useful for EL extensions. Can you
please merge to 4.0 branch

-Shwetha


On Sat, Jul 20, 2013 at 5:36 AM, Rohini Palaniswamy  wrote:

> This is done now. Changed version to 4.0.0 for the following
>
> project = OOZIE AND fixVersion = trunk AND resolution = Fixed AND Key not
> in (OOZIE-1453, OOZIE-1447, OOZIE-1440) AND fixVersion not in ("3.3.0",
> "3.3.1", "3.3.2")
>
>   Also removed trunk from FixVersion/s when there were other versions.
>
> Regards,
> Rohini
>
> On Sat, Jul 13, 2013 at 6:40 PM, Rohini Palaniswamy <
> rohini.adi...@gmail.com
> > wrote:
>
> > > Do you mean the "Fix Version" field?
> >Yes.
> >
> > >  I know that many JIRAs are using that
> > to also mean the "target version", which is a field we don't have
> >  "Fix Version" is the equivalent of bugzilla "Target Version".
> >
> > >  I'm sure there are some JIRAs marked with
> > "Fix Version" as trunk that we don't want in 4.0, right?
> >   Yes. A couple of jiras went into trunk after the branching. They will
> > not be marked 4.0
> >
> > Regards,
> > Rohini
> >
> >
> > On Fri, Jul 12, 2013 at 2:10 PM, Robert Kanter  >wrote:
> >
> >> Do you mean the "Fix Version" field?  I know that many JIRAs are using
> >> that
> >> to also mean the "target version", which is a field we don't have
> (perhaps
> >> we should have that field?).  I'm sure there are some JIRAs marked with
> >> "Fix Version" as trunk that we don't want in 4.0, right?
> >>
> >> - Robert
> >>
> >>
> >> On Fri, Jul 12, 2013 at 11:45 AM, Rohini Palaniswamy <
> >> rohini.adi...@gmail.com> wrote:
> >>
> >> > Planning to do a bulk update of the Fixed version of jiras to
> branch-4.0
> >> > tomorrow. Currently they are marked as trunk. Please let know if there
> >> are
> >> > any objections.
> >> >
> >> > Regards,
> >> > Rohini
> >> >
> >> > On Fri, Jun 28, 2013 at 10:27 PM, Rohini Palaniswamy <
> >> > rohini.adi...@gmail.com> wrote:
> >> >
> >> > > Created http://svn.apache.org/viewvc/oozie/branches/branch-4.0/.
> >> > >
> >> > > Patches yet to be done for branch-4.0.
> >> > >
> >> > >  OOZIE-1379 Generate SLA end_miss event only after confirming
> against
> >> > > persistent store
> >> > > OOZIE-1249 and OOZIE-1430 Documentation for JMS notifications and
> SLA
> >> > > monitoring
> >> > > OOZIE-1405 Fix flakey unit tests
> >> > >
> >> > > Regards,
> >> > > Rohini
> >> > >
> >> > > On Fri, Jun 28, 2013 at 11:41 AM, Robert Kanter <
> rkan...@cloudera.com
> >> > >wrote:
> >> > >
> >> > >> Ah, you're right; thanks.
> >> > >>
> >> > >> I reviewed the patch, but I'd like someone else to also take a look
> >> > >> because
> >> > >> I'm not a DB expert.
> >> > >>
> >> > >> - Robert
> >> > >>
> >> > >>
> >> > >> On Fri, Jun 28, 2013 at 11:30 AM, Virag Kothari <
> vi...@yahoo-inc.com
> >> >
> >> > >> wrote:
> >> > >>
> >> > >> > That change is reverted by OOZIE-1436. Can you review
> >> > >> > https://reviews.apache.org/r/12164/ and confirm?
> >> > >> > We can keep OOZIE-1437 open to deal the problems we will face
> when
> >> > >> having
> >> > >> > a 'proper' fix for the clob/varchar issues.
> >> > >> >
> >> > >> > Thanks,
> >> > >> > Virag
> >> > >> >
> >> > >> > On 6/28/13 11:12 AM, "Robert Kanter" 
> wrote:
> >> > >> >
> >> > >> > >I think we need both; OOZIE-1436 reverts the columns back
> because
> >> of
> >> > an
> >> > >> > >OpenJPA/Oracle issue and OOZIE-1437 is to fix an if statement
> >> added
> >> > >> into
> >> > >> > >our code (see
> >> > >> > >
> >> > >> >
> >> > >>
> >> >
> >>
> https://issues.apache.org/jira/browse/OOZIE-1398?focusedCommentId=13694224
> >> > >> >
> >> > >>
> >> >
> >>
> >&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#com
> >> > >> > >ment-13694224
> >> > >> > >)
> >> > >> > >
> >> > >> > >thanks
> >> > >> > >- Robert
> >> > >> > >
> >> > >> > >
> >> > >> > >On Fri, Jun 28, 2013 at 11:00 AM, Virag Kothari <
> >> vi...@yahoo-inc.com
> >> > >
> >> > >> > >wrote:
> >> > >> > >
> >> > >> > >> OOZIE-1436 will convert data varchar column back to lob due to
> >> some
> >> > >> > >>issues
> >> > >> > >> seen (In JIRA description) with OpenJPA.
> >> > >> > >>
> >> > >> > >>
> >> > >> > >> Thanks,
> >> > >> > >> Virag
> >> > >> > >>
> >> > >> > >> On 6/28/13 10:55 AM, "Robert Kanter" 
> >> wrote:
> >> > >> > >>
> >> > >> > >> >Can we also get OOZIE-1437 (paramerize size limit of setData
> in
> >> > >> > >> >JsonWorkflowAction based on user's configuration) in
> >> branch-4.0?
> >> > >> > >>Without
> >> > >> > >> >this, the size of  is always hardcoded at
> max
> >> of
> >> > >> 2k,
> >> > >> > >>even
> >> > >> > >> >if the user changes it in oozie-site (because of OOZIE-1398).
> >> > >> > >> >
> >> > >> > >> >thanks
> >> > >> > >> >- Robert
> >> > >> > >> >
> >> > >> > >> >
> >> > >> > >> >On Fri, Jun 28, 2013 at 9:21 AM, Rohini Palaniswamy
> >> > >> > >> > >> > >> > >> >> wrote:
> >> > >> > >> >
> >> > >> > >> >> Hi,
> >> > >> > >>

Review Request: [OOZIE-674] resolveInstanceRange doesn't work for EL extensions

[shwetha . gs]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/9065/
---

Review request for oozie.


Description
---

I have an EL extension today(0,0) which maps to start day of nominal time. This 
is used to specify startInstance, endInstance and instance in dataIn and 
dataOut of coordinator.
In CoordCommandUtils.resolveInstanceRange(), getInstanceNumber has to return 
the instance number with respect to current. So, for coord-action-create-inst 
context, I have mapped today to current and hence getInstanceNumber returns the 
correct number. But later in resolveInstanceRange(), getFuncType is called with 
startInstance value which is today in this case and it maps to UNEXPECTED and 
throws up. getFuncType should be passed the evaluation of 
coord-action-create-inst context


This addresses bug OOZIE-674.
https://issues.apache.org/jira/browse/OOZIE-674


Diffs
-

  
trunk/core/src/main/java/org/apache/oozie/command/coord/CoordCommandUtils.java 
1437311 
  trunk/core/src/main/java/org/apache/oozie/coord/CoordELFunctions.java 1437311 
  trunk/core/src/test/java/org/apache/oozie/command/coord/CoordELExtension.java 
PRE-CREATION 
  
trunk/core/src/test/java/org/apache/oozie/command/coord/TestCoordELExtensions.java
 PRE-CREATION 
  trunk/core/src/test/java/org/apache/oozie/test/XDataTestCase.java 1437311 
  trunk/core/src/test/java/org/apache/oozie/test/XTestCase.java 1437311 
  trunk/core/src/test/resources/coord-job-for-elext.xml PRE-CREATION 
  trunk/core/src/test/resources/oozie-site-coordel.xml PRE-CREATION 

Diff: https://reviews.apache.org/r/9065/diff/


Testing
---

Unit tests added


Thanks,

shwethags