[jira] [Commented] (OOZIE-1379) Generate SLA end_miss event only after confirming against persistent store

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-1379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13696067#comment-13696067
 ] 

Hadoop QA commented on OOZIE-1379:
--

Testing JIRA OOZIE-1379

Cleaning local svn workspace



{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:green}+1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than 
132
.{color:green}+1{color} the patch does adds/modifies 13 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT 
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc 
warnings
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac 
warnings
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA 
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:red}-1 TESTS{color}
.Tests run: 1255
.Tests failed: 3
.Tests errors: 5

.The patch failed the following testcases:

.  testStreamLog4JobLogDate(org.apache.oozie.TestCoordinatorEngineStreamLog)
.  testBundleKill2(org.apache.oozie.command.bundle.TestBundleKillXCommand)
.  
testCoordKillRemovePushMissingDeps(org.apache.oozie.command.coord.TestCoordKillXCommand)

{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch 


{color:red}*-1 Overall result, please check the reported -1(s)*{color}


The full output of the test-patch run is available at

.   https://builds.apache.org/job/oozie-trunk-precommit-build/638/

> Generate SLA end_miss event only after confirming against persistent store
> --
>
> Key: OOZIE-1379
> URL: https://issues.apache.org/jira/browse/OOZIE-1379
> Project: Oozie
>  Issue Type: Sub-task
>Reporter: Mona Chitnis
>Assignee: Mona Chitnis
> Attachments: OOZIE-1379.patch, OOZIE-1379.patch
>
>
> 1. SLA status "MISS" which occurs on an END_MISS i.e. job overshot expected 
> end time, is the most important sla notification for the user. We should make 
> sure against the source of truth i.e. database, that the scenario is indeed 
> correct for an end_miss, before setting the SLA status and sending out alerts 
> for it
> 2. hanging job id getter and setter in SLASummaryBean to getId and setId to 
> be consistent with getter in SLARegistrationBean
> 3. Fix bug where when Coordinator action goes from WAITING->KILLED directly, 
> no event is received

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Re: Branching for Oozie 4.0

[Rohini Palaniswamy]

Created http://svn.apache.org/viewvc/oozie/branches/branch-4.0/.

Patches yet to be done for branch-4.0.

 OOZIE-1379 Generate SLA end_miss event only after confirming against
persistent store
OOZIE-1249 and OOZIE-1430 Documentation for JMS notifications and SLA
monitoring
OOZIE-1405 Fix flakey unit tests

Regards,
Rohini

On Fri, Jun 28, 2013 at 11:41 AM, Robert Kanter wrote:

> Ah, you're right; thanks.
>
> I reviewed the patch, but I'd like someone else to also take a look because
> I'm not a DB expert.
>
> - Robert
>
>
> On Fri, Jun 28, 2013 at 11:30 AM, Virag Kothari 
> wrote:
>
> > That change is reverted by OOZIE-1436. Can you review
> > https://reviews.apache.org/r/12164/ and confirm?
> > We can keep OOZIE-1437 open to deal the problems we will face when having
> > a 'proper' fix for the clob/varchar issues.
> >
> > Thanks,
> > Virag
> >
> > On 6/28/13 11:12 AM, "Robert Kanter"  wrote:
> >
> > >I think we need both; OOZIE-1436 reverts the columns back because of an
> > >OpenJPA/Oracle issue and OOZIE-1437 is to fix an if statement added into
> > >our code (see
> > >
> >
> https://issues.apache.org/jira/browse/OOZIE-1398?focusedCommentId=13694224
> >
> >&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#com
> > >ment-13694224
> > >)
> > >
> > >thanks
> > >- Robert
> > >
> > >
> > >On Fri, Jun 28, 2013 at 11:00 AM, Virag Kothari 
> > >wrote:
> > >
> > >> OOZIE-1436 will convert data varchar column back to lob due to some
> > >>issues
> > >> seen (In JIRA description) with OpenJPA.
> > >>
> > >>
> > >> Thanks,
> > >> Virag
> > >>
> > >> On 6/28/13 10:55 AM, "Robert Kanter"  wrote:
> > >>
> > >> >Can we also get OOZIE-1437 (paramerize size limit of setData in
> > >> >JsonWorkflowAction based on user's configuration) in branch-4.0?
> > >>Without
> > >> >this, the size of  is always hardcoded at max of 2k,
> > >>even
> > >> >if the user changes it in oozie-site (because of OOZIE-1398).
> > >> >
> > >> >thanks
> > >> >- Robert
> > >> >
> > >> >
> > >> >On Fri, Jun 28, 2013 at 9:21 AM, Rohini Palaniswamy
> > >> > > >> >> wrote:
> > >> >
> > >> >> Hi,
> > >> >> I am planning to branch trunk as branch-4.0 and trash the
> current
> > >> >> branch-4.0 today evening. Discussed this with some of the folks
> > >>during
> > >> >>the
> > >> >> summit. This will facilitate releasing HCat integration, JMS
> > >> >>notifications
> > >> >> for job and SLA and SLA monitoring as Oozie 4.0 which we are done
> > >>with
> > >> >> functional testing and in the process of stress testing. This will
> > >>free
> > >> >>up
> > >> >> the trunk for the next release features of HA, cron based
> > >>scheduling, DB
> > >> >> clob column removal to move forward.
> > >> >>
> > >> >>
> > >> >> Patches that need to get into branch-4.0 are
> > >> >>
> > >> >> OOZIE-1433 ActionCheckX should override XCommand.getKey() to
> prevent
> > >> >> duplicates
> > >> >> OOZIE-1436 Revert SLA_XML and few other varchar columns back to
> clob
> > >> >> OOZIE-1427 Update CredentialsModule docs to mention Hive
> > >> >> OOZIE-1379 Generate SLA end_miss event only after confirming
> against
> > >> >> persistent store
> > >> >> OOZIE-1435 StatusTransitService unnecessarily updates the
> > >> >>lastModifiedTime
> > >> >> of jobs which causes MaterializationService to bring same jobs in
> > >>memory
> > >> >> OOZIE-1249 and OOZIE-1430 Documentation for JMS notifications and
> SLA
> > >> >> monitoring
> > >> >> Subworkflow notifications don't have workflow as parent id
> > >> >> And any other issues that crop up during the stress testing.
> > >> >>
> > >> >>Hoping most of this will get committed today before we branch.
> > >>Please
> > >> >> bring up any other jiras that we need to get into branch-4 before
> > >>moving
> > >> >> towards a release. Looking at a timeline of releasing 4.0 in 3
> weeks.
> > >> >>
> > >> >> Regards,
> > >> >> Rohini
> > >> >>
> > >>
> > >>
> >
> >
>

[jira] [Commented] (OOZIE-1379) Generate SLA end_miss event only after confirming against persistent store

[Virag Kothari (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-1379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13696027#comment-13696027
 ] 

Virag Kothari commented on OOZIE-1379:
--

+1 pending jenkins

> Generate SLA end_miss event only after confirming against persistent store
> --
>
> Key: OOZIE-1379
> URL: https://issues.apache.org/jira/browse/OOZIE-1379
> Project: Oozie
>  Issue Type: Sub-task
>Reporter: Mona Chitnis
>Assignee: Mona Chitnis
> Attachments: OOZIE-1379.patch, OOZIE-1379.patch
>
>
> 1. SLA status "MISS" which occurs on an END_MISS i.e. job overshot expected 
> end time, is the most important sla notification for the user. We should make 
> sure against the source of truth i.e. database, that the scenario is indeed 
> correct for an end_miss, before setting the SLA status and sending out alerts 
> for it
> 2. hanging job id getter and setter in SLASummaryBean to getId and setId to 
> be consistent with getter in SLARegistrationBean
> 3. Fix bug where when Coordinator action goes from WAITING->KILLED directly, 
> no event is received

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Re: Review Request 11797: OOZIE-1379 Generate SLA end_miss event only after confirming against persistent store

[Virag Kothari]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/11797/#review22560
---

Ship it!


Ship It!

- Virag Kothari


On June 29, 2013, 1:53 a.m., Mona Chitnis wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/11797/
> ---
> 
> (Updated June 29, 2013, 1:53 a.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-1379
> https://issues.apache.org/jira/browse/OOZIE-1379
> 
> 
> Repository: oozie
> 
> 
> Description
> ---
> 
> https://issues.apache.org/jira/browse/OOZIE-1379
> 
> 
> Diffs
> -
> 
>   trunk/core/src/main/java/org/apache/oozie/CoordinatorActionBean.java 
> 1497970 
>   trunk/core/src/main/java/org/apache/oozie/WorkflowActionBean.java 1497970 
>   trunk/core/src/main/java/org/apache/oozie/command/TransitionXCommand.java 
> 1497970 
>   
> trunk/core/src/main/java/org/apache/oozie/command/coord/CoordKillXCommand.java
>  1497970 
>   
> trunk/core/src/main/java/org/apache/oozie/command/wf/ActionStartXCommand.java 
> 1497970 
>   trunk/core/src/main/java/org/apache/oozie/event/BundleJobEvent.java 1497970 
>   trunk/core/src/main/java/org/apache/oozie/event/CoordinatorActionEvent.java 
> 1497970 
>   trunk/core/src/main/java/org/apache/oozie/event/CoordinatorJobEvent.java 
> 1497970 
>   trunk/core/src/main/java/org/apache/oozie/event/WorkflowActionEvent.java 
> 1497970 
>   trunk/core/src/main/java/org/apache/oozie/event/WorkflowJobEvent.java 
> 1497970 
>   
> trunk/core/src/main/java/org/apache/oozie/executor/jpa/sla/SLASummaryUpdateForSLAStatusActualTimesJPAExecutor.java
>  1497970 
>   
> trunk/core/src/main/java/org/apache/oozie/executor/jpa/sla/SLASummaryUpdateForSLAStatusJPAExecutor.java
>  1497970 
>   trunk/core/src/main/java/org/apache/oozie/sla/SLACalcStatus.java 1497970 
>   trunk/core/src/main/java/org/apache/oozie/sla/SLACalculator.java 1497970 
>   trunk/core/src/main/java/org/apache/oozie/sla/SLACalculatorMemory.java 
> 1497970 
>   trunk/core/src/main/java/org/apache/oozie/sla/SLAOperations.java 1497970 
>   trunk/core/src/main/java/org/apache/oozie/sla/SLARegistrationBean.java 
> 1497970 
>   trunk/core/src/main/java/org/apache/oozie/sla/SLASummaryBean.java 1497970 
>   trunk/core/src/test/java/org/apache/oozie/event/TestEventGeneration.java 
> 1497970 
>   trunk/core/src/test/java/org/apache/oozie/jms/TestJMSSLAEventListener.java 
> 1497970 
>   trunk/core/src/test/java/org/apache/oozie/servlet/TestV2SLAServlet.java 
> 1497970 
>   
> trunk/core/src/test/java/org/apache/oozie/sla/TestSLACalculationJPAExecutor.java
>  1497970 
>   trunk/core/src/test/java/org/apache/oozie/sla/TestSLACalculatorMemory.java 
> 1497970 
>   
> trunk/core/src/test/java/org/apache/oozie/sla/TestSLAEmailEventListener.java 
> 1497970 
>   trunk/core/src/test/java/org/apache/oozie/sla/TestSLAEventGeneration.java 
> 1497970 
>   trunk/core/src/test/java/org/apache/oozie/sla/TestSLAJobEventListener.java 
> 1497970 
>   
> trunk/core/src/test/java/org/apache/oozie/sla/TestSLARegistrationGetJPAExecutor.java
>  1497970 
>   
> trunk/core/src/test/java/org/apache/oozie/sla/TestSLARegistrationGetRecordsOnRestartJPAExecutor.java
>  1497970 
>   trunk/core/src/test/java/org/apache/oozie/sla/TestSLAService.java 1497970 
>   
> trunk/core/src/test/java/org/apache/oozie/sla/TestSLASummaryGetOnRestartJPAExecutor.java
>  1497970 
>   trunk/core/src/test/resources/coord-action-sla1.xml PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/11797/diff/
> 
> 
> Testing
> ---
> 
> added unit test
> 
> 
> Thanks,
> 
> Mona Chitnis
> 
>

[jira] [Updated] (OOZIE-1379) Generate SLA end_miss event only after confirming against persistent store

[Mona Chitnis (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-1379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mona Chitnis updated OOZIE-1379:


Attachment: (was: OOZIE-1379.patch)

> Generate SLA end_miss event only after confirming against persistent store
> --
>
> Key: OOZIE-1379
> URL: https://issues.apache.org/jira/browse/OOZIE-1379
> Project: Oozie
>  Issue Type: Sub-task
>Reporter: Mona Chitnis
>Assignee: Mona Chitnis
> Attachments: OOZIE-1379.patch, OOZIE-1379.patch
>
>
> 1. SLA status "MISS" which occurs on an END_MISS i.e. job overshot expected 
> end time, is the most important sla notification for the user. We should make 
> sure against the source of truth i.e. database, that the scenario is indeed 
> correct for an end_miss, before setting the SLA status and sending out alerts 
> for it
> 2. hanging job id getter and setter in SLASummaryBean to getId and setId to 
> be consistent with getter in SLARegistrationBean
> 3. Fix bug where when Coordinator action goes from WAITING->KILLED directly, 
> no event is received

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (OOZIE-1379) Generate SLA end_miss event only after confirming against persistent store

[Mona Chitnis (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-1379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mona Chitnis updated OOZIE-1379:


Attachment: OOZIE-1379.patch

> Generate SLA end_miss event only after confirming against persistent store
> --
>
> Key: OOZIE-1379
> URL: https://issues.apache.org/jira/browse/OOZIE-1379
> Project: Oozie
>  Issue Type: Sub-task
>Reporter: Mona Chitnis
>Assignee: Mona Chitnis
> Attachments: OOZIE-1379.patch, OOZIE-1379.patch
>
>
> 1. SLA status "MISS" which occurs on an END_MISS i.e. job overshot expected 
> end time, is the most important sla notification for the user. We should make 
> sure against the source of truth i.e. database, that the scenario is indeed 
> correct for an end_miss, before setting the SLA status and sending out alerts 
> for it
> 2. hanging job id getter and setter in SLASummaryBean to getId and setId to 
> be consistent with getter in SLARegistrationBean
> 3. Fix bug where when Coordinator action goes from WAITING->KILLED directly, 
> no event is received

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (OOZIE-1379) Generate SLA end_miss event only after confirming against persistent store

[Mona Chitnis (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-1379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mona Chitnis updated OOZIE-1379:


Attachment: OOZIE-1379.patch

> Generate SLA end_miss event only after confirming against persistent store
> --
>
> Key: OOZIE-1379
> URL: https://issues.apache.org/jira/browse/OOZIE-1379
> Project: Oozie
>  Issue Type: Sub-task
>Reporter: Mona Chitnis
>Assignee: Mona Chitnis
> Attachments: OOZIE-1379.patch, OOZIE-1379.patch
>
>
> 1. SLA status "MISS" which occurs on an END_MISS i.e. job overshot expected 
> end time, is the most important sla notification for the user. We should make 
> sure against the source of truth i.e. database, that the scenario is indeed 
> correct for an end_miss, before setting the SLA status and sending out alerts 
> for it
> 2. hanging job id getter and setter in SLASummaryBean to getId and setId to 
> be consistent with getter in SLARegistrationBean
> 3. Fix bug where when Coordinator action goes from WAITING->KILLED directly, 
> no event is received

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Re: Review Request 11797: OOZIE-1379 Generate SLA end_miss event only after confirming against persistent store

[Mona Chitnis]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/11797/
---

(Updated June 29, 2013, 1:53 a.m.)


Review request for oozie.


Changes
---

Addressed Virag's comments


Bugs: OOZIE-1379
https://issues.apache.org/jira/browse/OOZIE-1379


Repository: oozie


Description
---

https://issues.apache.org/jira/browse/OOZIE-1379


Diffs (updated)
-

  trunk/core/src/main/java/org/apache/oozie/CoordinatorActionBean.java 1497970 
  trunk/core/src/main/java/org/apache/oozie/WorkflowActionBean.java 1497970 
  trunk/core/src/main/java/org/apache/oozie/command/TransitionXCommand.java 
1497970 
  
trunk/core/src/main/java/org/apache/oozie/command/coord/CoordKillXCommand.java 
1497970 
  trunk/core/src/main/java/org/apache/oozie/command/wf/ActionStartXCommand.java 
1497970 
  trunk/core/src/main/java/org/apache/oozie/event/BundleJobEvent.java 1497970 
  trunk/core/src/main/java/org/apache/oozie/event/CoordinatorActionEvent.java 
1497970 
  trunk/core/src/main/java/org/apache/oozie/event/CoordinatorJobEvent.java 
1497970 
  trunk/core/src/main/java/org/apache/oozie/event/WorkflowActionEvent.java 
1497970 
  trunk/core/src/main/java/org/apache/oozie/event/WorkflowJobEvent.java 1497970 
  
trunk/core/src/main/java/org/apache/oozie/executor/jpa/sla/SLASummaryUpdateForSLAStatusActualTimesJPAExecutor.java
 1497970 
  
trunk/core/src/main/java/org/apache/oozie/executor/jpa/sla/SLASummaryUpdateForSLAStatusJPAExecutor.java
 1497970 
  trunk/core/src/main/java/org/apache/oozie/sla/SLACalcStatus.java 1497970 
  trunk/core/src/main/java/org/apache/oozie/sla/SLACalculator.java 1497970 
  trunk/core/src/main/java/org/apache/oozie/sla/SLACalculatorMemory.java 
1497970 
  trunk/core/src/main/java/org/apache/oozie/sla/SLAOperations.java 1497970 
  trunk/core/src/main/java/org/apache/oozie/sla/SLARegistrationBean.java 
1497970 
  trunk/core/src/main/java/org/apache/oozie/sla/SLASummaryBean.java 1497970 
  trunk/core/src/test/java/org/apache/oozie/event/TestEventGeneration.java 
1497970 
  trunk/core/src/test/java/org/apache/oozie/jms/TestJMSSLAEventListener.java 
1497970 
  trunk/core/src/test/java/org/apache/oozie/servlet/TestV2SLAServlet.java 
1497970 
  
trunk/core/src/test/java/org/apache/oozie/sla/TestSLACalculationJPAExecutor.java
 1497970 
  trunk/core/src/test/java/org/apache/oozie/sla/TestSLACalculatorMemory.java 
1497970 
  trunk/core/src/test/java/org/apache/oozie/sla/TestSLAEmailEventListener.java 
1497970 
  trunk/core/src/test/java/org/apache/oozie/sla/TestSLAEventGeneration.java 
1497970 
  trunk/core/src/test/java/org/apache/oozie/sla/TestSLAJobEventListener.java 
1497970 
  
trunk/core/src/test/java/org/apache/oozie/sla/TestSLARegistrationGetJPAExecutor.java
 1497970 
  
trunk/core/src/test/java/org/apache/oozie/sla/TestSLARegistrationGetRecordsOnRestartJPAExecutor.java
 1497970 
  trunk/core/src/test/java/org/apache/oozie/sla/TestSLAService.java 1497970 
  
trunk/core/src/test/java/org/apache/oozie/sla/TestSLASummaryGetOnRestartJPAExecutor.java
 1497970 
  trunk/core/src/test/resources/coord-action-sla1.xml PRE-CREATION 

Diff: https://reviews.apache.org/r/11797/diff/


Testing
---

added unit test


Thanks,

Mona Chitnis

Re: Review Request 11797: OOZIE-1379 Generate SLA end_miss event only after confirming against persistent store

[Mona Chitnis]

> On June 28, 2013, 11:37 p.m., Virag Kothari wrote:
> > trunk/core/src/test/java/org/apache/oozie/event/TestEventGeneration.java, 
> > line 553
> > 
> >
> > make sure this is not flaky

test doesnt fail even after run multiple times. If case of duplicate events (WF 
action) arises, will fix in another JIRA


- Mona


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/11797/#review22545
---


On June 28, 2013, 10:22 p.m., Mona Chitnis wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/11797/
> ---
> 
> (Updated June 28, 2013, 10:22 p.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-1379
> https://issues.apache.org/jira/browse/OOZIE-1379
> 
> 
> Repository: oozie
> 
> 
> Description
> ---
> 
> https://issues.apache.org/jira/browse/OOZIE-1379
> 
> 
> Diffs
> -
> 
>   trunk/core/src/main/java/org/apache/oozie/CoordinatorActionBean.java 
> 1497884 
>   trunk/core/src/main/java/org/apache/oozie/WorkflowActionBean.java 1497884 
>   trunk/core/src/main/java/org/apache/oozie/command/TransitionXCommand.java 
> 1497884 
>   
> trunk/core/src/main/java/org/apache/oozie/command/coord/CoordKillXCommand.java
>  1497884 
>   
> trunk/core/src/main/java/org/apache/oozie/command/wf/ActionStartXCommand.java 
> 1497884 
>   trunk/core/src/main/java/org/apache/oozie/event/BundleJobEvent.java 1497884 
>   trunk/core/src/main/java/org/apache/oozie/event/CoordinatorActionEvent.java 
> 1497884 
>   trunk/core/src/main/java/org/apache/oozie/event/CoordinatorJobEvent.java 
> 1497884 
>   trunk/core/src/main/java/org/apache/oozie/event/WorkflowActionEvent.java 
> 1497884 
>   trunk/core/src/main/java/org/apache/oozie/event/WorkflowJobEvent.java 
> 1497884 
>   
> trunk/core/src/main/java/org/apache/oozie/executor/jpa/sla/SLASummaryUpdateForSLAStatusActualTimesJPAExecutor.java
>  1497884 
>   
> trunk/core/src/main/java/org/apache/oozie/executor/jpa/sla/SLASummaryUpdateForSLAStatusJPAExecutor.java
>  1497884 
>   trunk/core/src/main/java/org/apache/oozie/sla/SLACalcStatus.java 1497884 
>   trunk/core/src/main/java/org/apache/oozie/sla/SLACalculator.java 1497884 
>   trunk/core/src/main/java/org/apache/oozie/sla/SLACalculatorMemory.java 
> 1497884 
>   trunk/core/src/main/java/org/apache/oozie/sla/SLAOperations.java 1497884 
>   trunk/core/src/main/java/org/apache/oozie/sla/SLARegistrationBean.java 
> 1497884 
>   trunk/core/src/main/java/org/apache/oozie/sla/SLASummaryBean.java 1497884 
>   trunk/core/src/test/java/org/apache/oozie/event/TestEventGeneration.java 
> 1497884 
>   trunk/core/src/test/java/org/apache/oozie/jms/TestJMSSLAEventListener.java 
> 1497884 
>   trunk/core/src/test/java/org/apache/oozie/servlet/TestV2SLAServlet.java 
> 1497884 
>   
> trunk/core/src/test/java/org/apache/oozie/sla/TestSLACalculationJPAExecutor.java
>  1497884 
>   trunk/core/src/test/java/org/apache/oozie/sla/TestSLACalculatorMemory.java 
> 1497884 
>   
> trunk/core/src/test/java/org/apache/oozie/sla/TestSLAEmailEventListener.java 
> 1497884 
>   trunk/core/src/test/java/org/apache/oozie/sla/TestSLAEventGeneration.java 
> 1497884 
>   trunk/core/src/test/java/org/apache/oozie/sla/TestSLAJobEventListener.java 
> 1497884 
>   
> trunk/core/src/test/java/org/apache/oozie/sla/TestSLARegistrationGetJPAExecutor.java
>  1497884 
>   
> trunk/core/src/test/java/org/apache/oozie/sla/TestSLARegistrationGetRecordsOnRestartJPAExecutor.java
>  1497884 
>   trunk/core/src/test/java/org/apache/oozie/sla/TestSLAService.java 1497884 
>   
> trunk/core/src/test/java/org/apache/oozie/sla/TestSLASummaryGetOnRestartJPAExecutor.java
>  1497884 
>   trunk/core/src/test/resources/coord-action-sla1.xml PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/11797/diff/
> 
> 
> Testing
> ---
> 
> added unit test
> 
> 
> Thanks,
> 
> Mona Chitnis
> 
>

[jira] [Commented] (OOZIE-1439) Job materialization happening even after coordinator is supposed to be killed.

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-1439?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13695979#comment-13695979
 ] 

Hadoop QA commented on OOZIE-1439:
--

Testing JIRA OOZIE-1439

Cleaning local svn workspace



{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:green}+1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than 
132
.{color:green}+1{color} the patch does adds/modifies 1 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT 
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc 
warnings
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac 
warnings
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA 
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:red}-1 TESTS{color}
.Tests run: 1252
.Tests failed: 0
.Tests errors: 1

.The patch failed the following testcases:

.  

{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch 


{color:red}*-1 Overall result, please check the reported -1(s)*{color}


The full output of the test-patch run is available at

.   https://builds.apache.org/job/oozie-trunk-precommit-build/637/

> Job materialization happening even after coordinator is supposed to be killed.
> --
>
> Key: OOZIE-1439
> URL: https://issues.apache.org/jira/browse/OOZIE-1439
> Project: Oozie
>  Issue Type: Bug
>Reporter: Virag Kothari
>Assignee: Virag Kothari
> Fix For: trunk
>
> Attachments: OOZIE-1439.patch
>
>
> In some cases where executon of StatusTransitService and CoordKill are 
> happening at same time, it may happen that the StatusTransitService 
> overwrites the KILLED state of job to RUNNING 
> This can be fixed by setting the doneMaterialization to true in CoordKillX
> and failing the materializtion if the job's doneMaterialization to true. 
> Also, eventually the StatusTransitService will convert the status back to 
> KILLED 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (OOZIE-1405) Fix flakey SLA tests

[Ryota Egashira (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-1405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13695967#comment-13695967
 ] 

Ryota Egashira commented on OOZIE-1405:
---

Other flaky test is TestSLAEventGeneration.testCoordinatorActionCommands
{code}
new CoordKillXCommand(jobId).call();
ehs.new EventWorker().run();
ehs.getEventQueue().poll(); //ignore duration event
slaEvent = (SLACalcStatus) ehs.getEventQueue().poll();
{code}
slaEvent is null, which cause exception. reason is, actions have been already 
in terminal state, and no actions killed in CoordKillXCommand. hence no new 
jobevent message pushed to queue.

> Fix flakey SLA tests
> 
>
> Key: OOZIE-1405
> URL: https://issues.apache.org/jira/browse/OOZIE-1405
> Project: Oozie
>  Issue Type: Sub-task
>Reporter: Rohini Palaniswamy
>Assignee: Rohini Palaniswamy
> Fix For: trunk
>
> Attachments: 1405-part.patch
>
>
> The SLA tests keep failing randomly because they rely on Thread.sleep. Would 
> be better to execute all test case scenarios and then assert everything in 
> order from eventQueue to avoid flakey failures.
> Similar for time-sensitive TestEventGeneration tests e.g. - 
> org.apache.oozie.event.TestEventGeneration.testCoordinatorActionEvent

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (OOZIE-1438) parentID is null in job event message of subworkflow action

[Ryota Egashira (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-1438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13695953#comment-13695953
 ] 

Ryota Egashira commented on OOZIE-1438:
---

test failure from flaky test cases, will fix in other JIRA

> parentID is null in job event message of subworkflow action 
> 
>
> Key: OOZIE-1438
> URL: https://issues.apache.org/jira/browse/OOZIE-1438
> Project: Oozie
>  Issue Type: Bug
>Affects Versions: trunk
>Reporter: Ryota Egashira
>Assignee: Ryota Egashira
> Attachments: OOZIE-1438.patch, OOZIE-1438-v2.patch
>
>
> this fix is required for subworkflow action where currently parentID 
> overwritten to null

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Re: Review Request 11797: OOZIE-1379 Generate SLA end_miss event only after confirming against persistent store

[Virag Kothari]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/11797/#review22545
---



trunk/core/src/main/java/org/apache/oozie/sla/SLACalculatorMemory.java


need to retrieve from both slasummary and registration



trunk/core/src/main/java/org/apache/oozie/sla/SLACalculatorMemory.java


if SLASummaryUpdateForSLAStatusJPAExecutor is no longer used, delete the 
file



trunk/core/src/main/java/org/apache/oozie/sla/SLACalculatorMemory.java


need to set ended=true here



trunk/core/src/main/java/org/apache/oozie/sla/SLACalculatorMemory.java


processDuration will overwrite the END event with DUration event. Move this 
if block above isMiss block, that will fix it it seems. Can we have a testcase?



trunk/core/src/test/java/org/apache/oozie/event/TestEventGeneration.java


make sure this is not flaky



trunk/core/src/test/java/org/apache/oozie/sla/TestSLAService.java


check the eventProcessed byte is set correctly, also check if job is 
removed from memory.


- Virag Kothari


On June 28, 2013, 10:22 p.m., Mona Chitnis wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/11797/
> ---
> 
> (Updated June 28, 2013, 10:22 p.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-1379
> https://issues.apache.org/jira/browse/OOZIE-1379
> 
> 
> Repository: oozie
> 
> 
> Description
> ---
> 
> https://issues.apache.org/jira/browse/OOZIE-1379
> 
> 
> Diffs
> -
> 
>   trunk/core/src/main/java/org/apache/oozie/CoordinatorActionBean.java 
> 1497884 
>   trunk/core/src/main/java/org/apache/oozie/WorkflowActionBean.java 1497884 
>   trunk/core/src/main/java/org/apache/oozie/command/TransitionXCommand.java 
> 1497884 
>   
> trunk/core/src/main/java/org/apache/oozie/command/coord/CoordKillXCommand.java
>  1497884 
>   
> trunk/core/src/main/java/org/apache/oozie/command/wf/ActionStartXCommand.java 
> 1497884 
>   trunk/core/src/main/java/org/apache/oozie/event/BundleJobEvent.java 1497884 
>   trunk/core/src/main/java/org/apache/oozie/event/CoordinatorActionEvent.java 
> 1497884 
>   trunk/core/src/main/java/org/apache/oozie/event/CoordinatorJobEvent.java 
> 1497884 
>   trunk/core/src/main/java/org/apache/oozie/event/WorkflowActionEvent.java 
> 1497884 
>   trunk/core/src/main/java/org/apache/oozie/event/WorkflowJobEvent.java 
> 1497884 
>   
> trunk/core/src/main/java/org/apache/oozie/executor/jpa/sla/SLASummaryUpdateForSLAStatusActualTimesJPAExecutor.java
>  1497884 
>   
> trunk/core/src/main/java/org/apache/oozie/executor/jpa/sla/SLASummaryUpdateForSLAStatusJPAExecutor.java
>  1497884 
>   trunk/core/src/main/java/org/apache/oozie/sla/SLACalcStatus.java 1497884 
>   trunk/core/src/main/java/org/apache/oozie/sla/SLACalculator.java 1497884 
>   trunk/core/src/main/java/org/apache/oozie/sla/SLACalculatorMemory.java 
> 1497884 
>   trunk/core/src/main/java/org/apache/oozie/sla/SLAOperations.java 1497884 
>   trunk/core/src/main/java/org/apache/oozie/sla/SLARegistrationBean.java 
> 1497884 
>   trunk/core/src/main/java/org/apache/oozie/sla/SLASummaryBean.java 1497884 
>   trunk/core/src/test/java/org/apache/oozie/event/TestEventGeneration.java 
> 1497884 
>   trunk/core/src/test/java/org/apache/oozie/jms/TestJMSSLAEventListener.java 
> 1497884 
>   trunk/core/src/test/java/org/apache/oozie/servlet/TestV2SLAServlet.java 
> 1497884 
>   
> trunk/core/src/test/java/org/apache/oozie/sla/TestSLACalculationJPAExecutor.java
>  1497884 
>   trunk/core/src/test/java/org/apache/oozie/sla/TestSLACalculatorMemory.java 
> 1497884 
>   
> trunk/core/src/test/java/org/apache/oozie/sla/TestSLAEmailEventListener.java 
> 1497884 
>   trunk/core/src/test/java/org/apache/oozie/sla/TestSLAEventGeneration.java 
> 1497884 
>   trunk/core/src/test/java/org/apache/oozie/sla/TestSLAJobEventListener.java 
> 1497884 
>   
> trunk/core/src/test/java/org/apache/oozie/sla/TestSLARegistrationGetJPAExecutor.java
>  1497884 
>   
> trunk/core/src/test/java/org/apache/oozie/sla/TestSLARegistrationGetRecordsOnRestartJPAExecutor.java
>  1497884 
>   trunk/core/src/test/java/org/apache/oozie/sla/TestSLAService.java 1497884 
>   
> trunk/core/src/test/java/org/apache/oozie/sla/TestSLASummaryGetOnRestartJPAExecutor.java
>  1497884 
>   trunk/core/src/test/resources/coord-action-sla1.xml PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/11797/diff/
> 
> 
> Testing
> ---
> 
> added unit test
> 
> 
> Thanks,
> 
> M

[jira] [Commented] (OOZIE-1438) parentID is null in job event message of subworkflow action

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-1438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13695921#comment-13695921
 ] 

Hadoop QA commented on OOZIE-1438:
--

Testing JIRA OOZIE-1438

Cleaning local svn workspace



{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:red}-1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than 
132
.{color:red}-1{color} the patch does not add/modify any testcase
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT 
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc 
warnings
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac 
warnings
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA 
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:red}-1 TESTS{color}
.Tests run: 1252
.Tests failed: 0
.Tests errors: 2

.The patch failed the following testcases:

.  

{color:red}-1 DISTRO{color}
.{color:red}-1{color} distro tarball fails with the patch


{color:red}*-1 Overall result, please check the reported -1(s)*{color}


The full output of the test-patch run is available at

.   https://builds.apache.org/job/oozie-trunk-precommit-build/636/

> parentID is null in job event message of subworkflow action 
> 
>
> Key: OOZIE-1438
> URL: https://issues.apache.org/jira/browse/OOZIE-1438
> Project: Oozie
>  Issue Type: Bug
>Affects Versions: trunk
>Reporter: Ryota Egashira
>Assignee: Ryota Egashira
> Attachments: OOZIE-1438.patch, OOZIE-1438-v2.patch
>
>
> this fix is required for subworkflow action where currently parentID 
> overwritten to null

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (OOZIE-1435) StatusTransitService unnecessarily updates the lastModifiedTime of jobs which causes MaterializationService to bring same jobs in memory

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-1435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13695868#comment-13695868
 ] 

Hadoop QA commented on OOZIE-1435:
--

Testing JIRA OOZIE-1435

Cleaning local svn workspace



{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:green}+1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than 
132
.{color:green}+1{color} the patch does adds/modifies 1 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT 
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc 
warnings
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac 
warnings
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA 
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:red}-1 TESTS{color}
.Tests run: 1253
.Tests failed: 0
.Tests errors: 2

.The patch failed the following testcases:

.  

{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch 


{color:red}*-1 Overall result, please check the reported -1(s)*{color}


The full output of the test-patch run is available at

.   https://builds.apache.org/job/oozie-trunk-precommit-build/635/

> StatusTransitService unnecessarily updates the lastModifiedTime of jobs which 
> causes MaterializationService to bring same jobs in memory
> 
>
> Key: OOZIE-1435
> URL: https://issues.apache.org/jira/browse/OOZIE-1435
> Project: Oozie
>  Issue Type: Bug
>Reporter: Virag Kothari
>Assignee: Virag Kothari
> Fix For: trunk
>
> Attachments: OOZIE-1435.patch, OOZIE-1435.patch, OOZIE-1435.patch
>
>
> CoordMaterializationTriggerService
> brings 50 actions for materializing every 5 mins based on last modified
> time. It updates the last modified time every time for each of this action so
> new jobs can get a chance in the next interval. However, StatusTransitService
> unnecessarily keeps updating the last modified time which breaks the logic of
> materialization service as it may keep on bringing the same actions for
> materialization every scheduled interval.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Re: Review Request 11797: OOZIE-1379 Generate SLA end_miss event only after confirming against persistent store

[Mona Chitnis]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/11797/
---

(Updated June 28, 2013, 10:22 p.m.)


Review request for oozie.


Changes
---

Virag's comments addressed. Changed SLACalculatorMemory.get() definition a bit, 
to fetch object related to history set jobId, when not found in map. Also 
edited and enhanced tests to check for actual times set in both cases of 
confirmWithDB Met/Miss


Bugs: OOZIE-1379
https://issues.apache.org/jira/browse/OOZIE-1379


Repository: oozie


Description
---

https://issues.apache.org/jira/browse/OOZIE-1379


Diffs (updated)
-

  trunk/core/src/main/java/org/apache/oozie/CoordinatorActionBean.java 1497884 
  trunk/core/src/main/java/org/apache/oozie/WorkflowActionBean.java 1497884 
  trunk/core/src/main/java/org/apache/oozie/command/TransitionXCommand.java 
1497884 
  
trunk/core/src/main/java/org/apache/oozie/command/coord/CoordKillXCommand.java 
1497884 
  trunk/core/src/main/java/org/apache/oozie/command/wf/ActionStartXCommand.java 
1497884 
  trunk/core/src/main/java/org/apache/oozie/event/BundleJobEvent.java 1497884 
  trunk/core/src/main/java/org/apache/oozie/event/CoordinatorActionEvent.java 
1497884 
  trunk/core/src/main/java/org/apache/oozie/event/CoordinatorJobEvent.java 
1497884 
  trunk/core/src/main/java/org/apache/oozie/event/WorkflowActionEvent.java 
1497884 
  trunk/core/src/main/java/org/apache/oozie/event/WorkflowJobEvent.java 1497884 
  
trunk/core/src/main/java/org/apache/oozie/executor/jpa/sla/SLASummaryUpdateForSLAStatusActualTimesJPAExecutor.java
 1497884 
  
trunk/core/src/main/java/org/apache/oozie/executor/jpa/sla/SLASummaryUpdateForSLAStatusJPAExecutor.java
 1497884 
  trunk/core/src/main/java/org/apache/oozie/sla/SLACalcStatus.java 1497884 
  trunk/core/src/main/java/org/apache/oozie/sla/SLACalculator.java 1497884 
  trunk/core/src/main/java/org/apache/oozie/sla/SLACalculatorMemory.java 
1497884 
  trunk/core/src/main/java/org/apache/oozie/sla/SLAOperations.java 1497884 
  trunk/core/src/main/java/org/apache/oozie/sla/SLARegistrationBean.java 
1497884 
  trunk/core/src/main/java/org/apache/oozie/sla/SLASummaryBean.java 1497884 
  trunk/core/src/test/java/org/apache/oozie/event/TestEventGeneration.java 
1497884 
  trunk/core/src/test/java/org/apache/oozie/jms/TestJMSSLAEventListener.java 
1497884 
  trunk/core/src/test/java/org/apache/oozie/servlet/TestV2SLAServlet.java 
1497884 
  
trunk/core/src/test/java/org/apache/oozie/sla/TestSLACalculationJPAExecutor.java
 1497884 
  trunk/core/src/test/java/org/apache/oozie/sla/TestSLACalculatorMemory.java 
1497884 
  trunk/core/src/test/java/org/apache/oozie/sla/TestSLAEmailEventListener.java 
1497884 
  trunk/core/src/test/java/org/apache/oozie/sla/TestSLAEventGeneration.java 
1497884 
  trunk/core/src/test/java/org/apache/oozie/sla/TestSLAJobEventListener.java 
1497884 
  
trunk/core/src/test/java/org/apache/oozie/sla/TestSLARegistrationGetJPAExecutor.java
 1497884 
  
trunk/core/src/test/java/org/apache/oozie/sla/TestSLARegistrationGetRecordsOnRestartJPAExecutor.java
 1497884 
  trunk/core/src/test/java/org/apache/oozie/sla/TestSLAService.java 1497884 
  
trunk/core/src/test/java/org/apache/oozie/sla/TestSLASummaryGetOnRestartJPAExecutor.java
 1497884 
  trunk/core/src/test/resources/coord-action-sla1.xml PRE-CREATION 

Diff: https://reviews.apache.org/r/11797/diff/


Testing
---

added unit test


Thanks,

Mona Chitnis

[jira] [Commented] (OOZIE-1439) Job materialization happening even after coordinator is supposed to be killed.

[Rohini Palaniswamy (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-1439?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13695858#comment-13695858
 ] 

Rohini Palaniswamy commented on OOZIE-1439:
---

+1

> Job materialization happening even after coordinator is supposed to be killed.
> --
>
> Key: OOZIE-1439
> URL: https://issues.apache.org/jira/browse/OOZIE-1439
> Project: Oozie
>  Issue Type: Bug
>Reporter: Virag Kothari
>Assignee: Virag Kothari
> Attachments: OOZIE-1439.patch
>
>
> In some cases where executon of StatusTransitService and CoordKill are 
> happening at same time, it may happen that the StatusTransitService 
> overwrites the KILLED state of job to RUNNING 
> This can be fixed by setting the doneMaterialization to true in CoordKillX
> and failing the materializtion if the job's doneMaterialization to true. 
> Also, eventually the StatusTransitService will convert the status back to 
> KILLED 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (OOZIE-1439) Job materialization happening even after coordinator is supposed to be killed.

[Virag Kothari (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-1439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Virag Kothari updated OOZIE-1439:
-

Attachment: OOZIE-1439.patch

> Job materialization happening even after coordinator is supposed to be killed.
> --
>
> Key: OOZIE-1439
> URL: https://issues.apache.org/jira/browse/OOZIE-1439
> Project: Oozie
>  Issue Type: Bug
>Reporter: Virag Kothari
>Assignee: Virag Kothari
> Attachments: OOZIE-1439.patch
>
>
> In some cases where executon of StatusTransitService and CoordKill are 
> happening at same time, it may happen that the StatusTransitService 
> overwrites the KILLED state of job to RUNNING 
> This can be fixed by setting the doneMaterialization to true in CoordKillX
> and failing the materializtion if the job's doneMaterialization to true. 
> Also, eventually the StatusTransitService will convert the status back to 
> KILLED 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (OOZIE-1438) parentID is null in job event message of subworkflow action

[Ryota Egashira (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-1438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ryota Egashira updated OOZIE-1438:
--

Description: this fix is required for subworkflow action where currently 
parentID overwritten to null

> parentID is null in job event message of subworkflow action 
> 
>
> Key: OOZIE-1438
> URL: https://issues.apache.org/jira/browse/OOZIE-1438
> Project: Oozie
>  Issue Type: Bug
>Affects Versions: trunk
>Reporter: Ryota Egashira
>Assignee: Ryota Egashira
> Attachments: OOZIE-1438.patch, OOZIE-1438-v2.patch
>
>
> this fix is required for subworkflow action where currently parentID 
> overwritten to null

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Re: Review Request 12177: [OOZIE-1438] parentID is null in job event message of subworkflow action

[Virag Kothari]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/12177/#review22538
---


+1.. Before committing add a comment for that if() that it is for subworkflow

- Virag Kothari


On June 28, 2013, 8:58 p.m., Ryota Egashira wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/12177/
> ---
> 
> (Updated June 28, 2013, 8:58 p.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-1438
> https://issues.apache.org/jira/browse/OOZIE-1438
> 
> 
> Repository: oozie
> 
> 
> Description
> ---
> 
> https://issues.apache.org/jira/browse/OOZIE-1438
> 
> 
> Diffs
> -
> 
>   trunk/core/src/main/java/org/apache/oozie/command/wf/SignalXCommand.java 
> 1497904 
> 
> Diff: https://reviews.apache.org/r/12177/diff/
> 
> 
> Testing
> ---
> 
> locally tested using single-node cluster
> 
> 
> Thanks,
> 
> Ryota Egashira
> 
>

[jira] [Updated] (OOZIE-1438) parentID is null in job event message of subworkflow action

[Ryota Egashira (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-1438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ryota Egashira updated OOZIE-1438:
--

Attachment: OOZIE-1438-v2.patch

> parentID is null in job event message of subworkflow action 
> 
>
> Key: OOZIE-1438
> URL: https://issues.apache.org/jira/browse/OOZIE-1438
> Project: Oozie
>  Issue Type: Bug
>Affects Versions: trunk
>Reporter: Ryota Egashira
>Assignee: Ryota Egashira
> Attachments: OOZIE-1438.patch, OOZIE-1438-v2.patch
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (OOZIE-1434) Commands not releasing memory lock

[Alejandro Abdelnur (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-1434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13695808#comment-13695808
 ] 

Alejandro Abdelnur commented on OOZIE-1434:
---

if the jdbc driver does not support it, then it should be something to 
configure on the SQLServer itself, if not there either, then there is no 
control from our side.

> Commands not releasing memory lock
> --
>
> Key: OOZIE-1434
> URL: https://issues.apache.org/jira/browse/OOZIE-1434
> Project: Oozie
>  Issue Type: Bug
>Reporter: Virag Kothari
>Assignee: Virag Kothari
>
> If the command gets stuck in making a db call during loadState() after 
> acquiring the lock, the lock will never be released and the job will be stuck 
> forever. 
> Need to introduce a timeout for the database call.
> There might be also be an issue with the implementation of MemoryLocks.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Re: Review Request 12177: [OOZIE-1438] parentID is null in job event message of subworkflow action

[Ryota Egashira]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/12177/#review22534
---


confirmed that, in case of coordinator and workflow with subwrk action,  
parentId is set properly in job event message

- Ryota Egashira


On June 28, 2013, 8:58 p.m., Ryota Egashira wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/12177/
> ---
> 
> (Updated June 28, 2013, 8:58 p.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-1438
> https://issues.apache.org/jira/browse/OOZIE-1438
> 
> 
> Repository: oozie
> 
> 
> Description
> ---
> 
> https://issues.apache.org/jira/browse/OOZIE-1438
> 
> 
> Diffs
> -
> 
>   trunk/core/src/main/java/org/apache/oozie/command/wf/SignalXCommand.java 
> 1497904 
> 
> Diff: https://reviews.apache.org/r/12177/diff/
> 
> 
> Testing
> ---
> 
> locally tested using single-node cluster
> 
> 
> Thanks,
> 
> Ryota Egashira
> 
>

[jira] [Commented] (OOZIE-1435) StatusTransitService unnecessarily updates the lastModifiedTime of jobs which causes MaterializationService to bring same jobs in memory

[Rohini Palaniswamy (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-1435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13695796#comment-13695796
 ] 

Rohini Palaniswamy commented on OOZIE-1435:
---

+1 Pending jenkins

> StatusTransitService unnecessarily updates the lastModifiedTime of jobs which 
> causes MaterializationService to bring same jobs in memory
> 
>
> Key: OOZIE-1435
> URL: https://issues.apache.org/jira/browse/OOZIE-1435
> Project: Oozie
>  Issue Type: Bug
>Reporter: Virag Kothari
>Assignee: Virag Kothari
> Fix For: trunk
>
> Attachments: OOZIE-1435.patch, OOZIE-1435.patch, OOZIE-1435.patch
>
>
> CoordMaterializationTriggerService
> brings 50 actions for materializing every 5 mins based on last modified
> time. It updates the last modified time every time for each of this action so
> new jobs can get a chance in the next interval. However, StatusTransitService
> unnecessarily keeps updating the last modified time which breaks the logic of
> materialization service as it may keep on bringing the same actions for
> materialization every scheduled interval.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (OOZIE-1435) StatusTransitService unnecessarily updates the lastModifiedTime of jobs which causes MaterializationService to bring same jobs in memory

[Virag Kothari (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-1435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Virag Kothari updated OOZIE-1435:
-

Attachment: OOZIE-1435.patch

Reattaching patch

> StatusTransitService unnecessarily updates the lastModifiedTime of jobs which 
> causes MaterializationService to bring same jobs in memory
> 
>
> Key: OOZIE-1435
> URL: https://issues.apache.org/jira/browse/OOZIE-1435
> Project: Oozie
>  Issue Type: Bug
>Reporter: Virag Kothari
>Assignee: Virag Kothari
> Fix For: trunk
>
> Attachments: OOZIE-1435.patch, OOZIE-1435.patch, OOZIE-1435.patch
>
>
> CoordMaterializationTriggerService
> brings 50 actions for materializing every 5 mins based on last modified
> time. It updates the last modified time every time for each of this action so
> new jobs can get a chance in the next interval. However, StatusTransitService
> unnecessarily keeps updating the last modified time which breaks the logic of
> materialization service as it may keep on bringing the same actions for
> materialization every scheduled interval.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (OOZIE-1438) parentID is null in job event message of subworkflow action

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-1438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13695789#comment-13695789
 ] 

Hadoop QA commented on OOZIE-1438:
--

Testing JIRA OOZIE-1438

Cleaning local svn workspace



{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:red}-1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than 
132
.{color:red}-1{color} the patch does not add/modify any testcase
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT 
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc 
warnings
{color:red}-1 COMPILE{color}
.{color:red}-1{color} HEAD does not compile
.{color:red}-1{color} patch does not compile
.{color:green}+1{color} the patch does not seem to introduce new javac 
warnings
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA 
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:red}-1 TESTS{color} - patch does not compile, cannot run testcases
{color:red}-1 DISTRO{color}
.{color:red}-1{color} distro tarball fails with the patch


{color:red}*-1 Overall result, please check the reported -1(s)*{color}


The full output of the test-patch run is available at

.   https://builds.apache.org/job/oozie-trunk-precommit-build/634/

> parentID is null in job event message of subworkflow action 
> 
>
> Key: OOZIE-1438
> URL: https://issues.apache.org/jira/browse/OOZIE-1438
> Project: Oozie
>  Issue Type: Bug
>Affects Versions: trunk
>Reporter: Ryota Egashira
>Assignee: Ryota Egashira
> Attachments: OOZIE-1438.patch
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (OOZIE-1435) StatusTransitService unnecessarily updates the lastModifiedTime of jobs which causes MaterializationService to bring same jobs in memory

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-1435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13695785#comment-13695785
 ] 

Hadoop QA commented on OOZIE-1435:
--

Testing JIRA OOZIE-1435

Cleaning local svn workspace



{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:green}+1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than 
132
.{color:green}+1{color} the patch does adds/modifies 1 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT 
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc 
warnings
{color:red}-1 COMPILE{color}
.{color:red}-1{color} HEAD does not compile
.{color:red}-1{color} patch does not compile
.{color:green}+1{color} the patch does not seem to introduce new javac 
warnings
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA 
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:red}-1 TESTS{color} - patch does not compile, cannot run testcases
{color:red}-1 DISTRO{color}
.{color:red}-1{color} distro tarball fails with the patch


{color:red}*-1 Overall result, please check the reported -1(s)*{color}


The full output of the test-patch run is available at

.   https://builds.apache.org/job/oozie-trunk-precommit-build/633/

> StatusTransitService unnecessarily updates the lastModifiedTime of jobs which 
> causes MaterializationService to bring same jobs in memory
> 
>
> Key: OOZIE-1435
> URL: https://issues.apache.org/jira/browse/OOZIE-1435
> Project: Oozie
>  Issue Type: Bug
>Reporter: Virag Kothari
>Assignee: Virag Kothari
> Fix For: trunk
>
> Attachments: OOZIE-1435.patch, OOZIE-1435.patch
>
>
> CoordMaterializationTriggerService
> brings 50 actions for materializing every 5 mins based on last modified
> time. It updates the last modified time every time for each of this action so
> new jobs can get a chance in the next interval. However, StatusTransitService
> unnecessarily keeps updating the last modified time which breaks the logic of
> materialization service as it may keep on bringing the same actions for
> materialization every scheduled interval.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Re: Review Request 12177: [OOZIE-1438] parentID is null in job event message of subworkflow action

[Ryota Egashira]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/12177/
---

(Updated June 28, 2013, 8:58 p.m.)


Review request for oozie.


Changes
---

talked offline with Virag, and fixed it, thanks.
now running examples (coordinator, workflow with subwrk action, etc) for final 
check


Bugs: OOZIE-1438
https://issues.apache.org/jira/browse/OOZIE-1438


Repository: oozie


Description
---

https://issues.apache.org/jira/browse/OOZIE-1438


Diffs (updated)
-

  trunk/core/src/main/java/org/apache/oozie/command/wf/SignalXCommand.java 
1497904 

Diff: https://reviews.apache.org/r/12177/diff/


Testing
---

locally tested using single-node cluster


Thanks,

Ryota Egashira

Re: Review Request 12177: [OOZIE-1438] parentID is null in job event message of subworkflow action

[Virag Kothari]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/12177/#review22532
---



trunk/core/src/main/java/org/apache/oozie/command/wf/SignalXCommand.java


Set parentId if not already set - discussed offline


- Virag Kothari


On June 28, 2013, 8:35 p.m., Ryota Egashira wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/12177/
> ---
> 
> (Updated June 28, 2013, 8:35 p.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-1438
> https://issues.apache.org/jira/browse/OOZIE-1438
> 
> 
> Repository: oozie
> 
> 
> Description
> ---
> 
> https://issues.apache.org/jira/browse/OOZIE-1438
> 
> 
> Diffs
> -
> 
>   trunk/core/src/main/java/org/apache/oozie/command/wf/SignalXCommand.java 
> 1497904 
> 
> Diff: https://reviews.apache.org/r/12177/diff/
> 
> 
> Testing
> ---
> 
> locally tested using single-node cluster
> 
> 
> Thanks,
> 
> Ryota Egashira
> 
>

Re: Review Request 11797: OOZIE-1379 Generate SLA end_miss event only after confirming against persistent store

[Mona Chitnis]

> On June 25, 2013, 4:35 p.m., Virag Kothari wrote:
> > trunk/core/src/main/java/org/apache/oozie/command/coord/CoordKillXCommand.java,
> >  line 117
> > 
> >
> > also add prevStatus!=TerminalState
> > 
> > It might happen that the workflow succeeds, so coord action succeeds 
> > but the coord job is not yet transitioned to success state. In this case, a 
> > coord_kill by user will generate a kill event even though the underlying 
> > workflow has succeeeded

This Coord Kill command is only loading actions which are non-terminal. Within 
the same command, it does not update action based on downstream workflow so why 
will this case occur?


> On June 25, 2013, 4:35 p.m., Virag Kothari wrote:
> > trunk/core/src/main/java/org/apache/oozie/sla/SLACalculatorMemory.java, 
> > line 339
> > 
> >
> > Also need to set actualDuration.

okay


> On June 25, 2013, 4:35 p.m., Virag Kothari wrote:
> > trunk/core/src/main/java/org/apache/oozie/sla/SLACalculatorMemory.java, 
> > line 342
> > 
> >
> > This JPA executor doesn't update the start, end etc. Use 
> > SLASummaryUpdateForSLAStatusActualTimesJPAExecutor
> > it will be good to have a testcase so such things are not missed
> >

thanks. adding lines in test case


> On June 25, 2013, 4:35 p.m., Virag Kothari wrote:
> > trunk/core/src/main/java/org/apache/oozie/sla/SLACalculatorMemory.java, 
> > line 705
> > 
> >
> > if the coordinator goes from WAITING-> KILLED, then this condition will 
> > be true.
> > Hence we need to set the startTime to null, and endTime to 
> > lastModifiedTime
> >

thanks and also added setting of actual start and end times for WF Job and 
Action at all times (because end_miss/end_met is being decided right after DB 
check and row updated in SLASummary)


> On June 25, 2013, 4:35 p.m., Virag Kothari wrote:
> > trunk/core/src/main/java/org/apache/oozie/sla/SLACalculatorMemory.java, 
> > line 741
> > 
> >
> > if a JPA exception is thrown, the end event processed bit will be set,
> > but neither MISS or MET event may be sent.
> > There seems two ways to fix this
> > 1) not set the event processed bit, so confirmWithDb will be called 
> > next time during periodic check
> > 2) Problem with 1) is a rogue job may cause JPA execption many times. 
> > To avoid that, we can just send out a MISS event as anyways the current 
> > clock has passed the expected end.

I also prefer option 2). thanks


- Mona


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/11797/#review22359
---


On June 25, 2013, 2:27 a.m., Mona Chitnis wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/11797/
> ---
> 
> (Updated June 25, 2013, 2:27 a.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-1379
> https://issues.apache.org/jira/browse/OOZIE-1379
> 
> 
> Repository: oozie
> 
> 
> Description
> ---
> 
> https://issues.apache.org/jira/browse/OOZIE-1379
> 
> 
> Diffs
> -
> 
>   trunk/core/src/main/java/org/apache/oozie/CoordinatorActionBean.java 
> 1496209 
>   trunk/core/src/main/java/org/apache/oozie/WorkflowActionBean.java 1496209 
>   trunk/core/src/main/java/org/apache/oozie/command/TransitionXCommand.java 
> 1496209 
>   
> trunk/core/src/main/java/org/apache/oozie/command/coord/CoordKillXCommand.java
>  1496209 
>   
> trunk/core/src/main/java/org/apache/oozie/command/wf/ActionStartXCommand.java 
> 1496209 
>   trunk/core/src/main/java/org/apache/oozie/event/BundleJobEvent.java 1496209 
>   trunk/core/src/main/java/org/apache/oozie/event/CoordinatorActionEvent.java 
> 1496209 
>   trunk/core/src/main/java/org/apache/oozie/event/CoordinatorJobEvent.java 
> 1496209 
>   trunk/core/src/main/java/org/apache/oozie/event/WorkflowActionEvent.java 
> 1496209 
>   trunk/core/src/main/java/org/apache/oozie/event/WorkflowJobEvent.java 
> 1496209 
>   
> trunk/core/src/main/java/org/apache/oozie/executor/jpa/sla/SLASummaryUpdateForSLAStatusActualTimesJPAExecutor.java
>  1496209 
>   
> trunk/core/src/main/java/org/apache/oozie/executor/jpa/sla/SLASummaryUpdateForSLAStatusJPAExecutor.java
>  1496209 
>   trunk/core/src/main/java/org/apache/oozie/sla/SLACalcStatus.java 1496209 
>   trunk/core/src/main/java/org/apache/oozie/sla/SLACalculator.java 1496209 
>   trunk/core/src/main/java/org/apache/oozie/sla/SLACalculatorMemory.java 
> 

[jira] [Updated] (OOZIE-1435) StatusTransitService unnecessarily updates the lastModifiedTime of jobs which causes MaterializationService to bring same jobs in memory

[Virag Kothari (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-1435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Virag Kothari updated OOZIE-1435:
-

Attachment: OOZIE-1435.patch

Attached patch fixes line > 132 and removes assertNotNull(JPAservice)

> StatusTransitService unnecessarily updates the lastModifiedTime of jobs which 
> causes MaterializationService to bring same jobs in memory
> 
>
> Key: OOZIE-1435
> URL: https://issues.apache.org/jira/browse/OOZIE-1435
> Project: Oozie
>  Issue Type: Bug
>Reporter: Virag Kothari
>Assignee: Virag Kothari
> Fix For: trunk
>
> Attachments: OOZIE-1435.patch, OOZIE-1435.patch
>
>
> CoordMaterializationTriggerService
> brings 50 actions for materializing every 5 mins based on last modified
> time. It updates the last modified time every time for each of this action so
> new jobs can get a chance in the next interval. However, StatusTransitService
> unnecessarily keeps updating the last modified time which breaks the logic of
> materialization service as it may keep on bringing the same actions for
> materialization every scheduled interval.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (OOZIE-1438) parentID is null in job event message of subworkflow action

[Ryota Egashira (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-1438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13695762#comment-13695762
 ] 

Ryota Egashira commented on OOZIE-1438:
---

https://reviews.apache.org/r/12177

> parentID is null in job event message of subworkflow action 
> 
>
> Key: OOZIE-1438
> URL: https://issues.apache.org/jira/browse/OOZIE-1438
> Project: Oozie
>  Issue Type: Bug
>Affects Versions: trunk
>Reporter: Ryota Egashira
>Assignee: Ryota Egashira
> Attachments: OOZIE-1438.patch
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (OOZIE-1438) parentID is null in job event message of subworkflow action

[Ryota Egashira (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-1438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ryota Egashira updated OOZIE-1438:
--

Attachment: OOZIE-1438.patch

> parentID is null in job event message of subworkflow action 
> 
>
> Key: OOZIE-1438
> URL: https://issues.apache.org/jira/browse/OOZIE-1438
> Project: Oozie
>  Issue Type: Bug
>Affects Versions: trunk
>Reporter: Ryota Egashira
>Assignee: Ryota Egashira
> Attachments: OOZIE-1438.patch
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Review Request 12177: [OOZIE-1438] parentID is null in job event message of subworkflow action

[Ryota Egashira]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/12177/
---

Review request for oozie.


Bugs: OOZIE-1438
https://issues.apache.org/jira/browse/OOZIE-1438


Repository: oozie


Description
---

https://issues.apache.org/jira/browse/OOZIE-1438


Diffs
-

  trunk/core/src/main/java/org/apache/oozie/command/wf/SignalXCommand.java 
1497904 

Diff: https://reviews.apache.org/r/12177/diff/


Testing
---

locally tested using single-node cluster


Thanks,

Ryota Egashira

Re: Review Request 12164: OOZIE-1436: Revert SLA_XML and few other varchar columns back to clob

[Ryota Egashira]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/12164/#review22529
---

Ship it!


- Ryota Egashira


On June 28, 2013, 5:03 p.m., Virag Kothari wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/12164/
> ---
> 
> (Updated June 28, 2013, 5:03 p.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-1436
> https://issues.apache.org/jira/browse/OOZIE-1436
> 
> 
> Repository: oozie
> 
> 
> Description
> ---
> 
> https://issues.apache.org/jira/browse/OOZIE-1436
> 
> Converting sla_xml column from wf_action, wf_job, coord_action, coord_job 
> tables back to clob. Also data column from wf_action converted back to clob
> 
> 
> Diffs
> -
> 
>   trunk/core/src/main/java/org/apache/oozie/CoordinatorActionBean.java 
> 1497646 
>   trunk/core/src/main/java/org/apache/oozie/CoordinatorJobBean.java 1497646 
>   trunk/core/src/main/java/org/apache/oozie/WorkflowActionBean.java 1497646 
>   trunk/core/src/main/java/org/apache/oozie/WorkflowJobBean.java 1497646 
>   
> trunk/core/src/main/java/org/apache/oozie/client/rest/JsonWorkflowAction.java 
> 1497646 
>   trunk/tools/src/main/java/org/apache/oozie/tools/OozieDBCLI.java 1497646 
> 
> Diff: https://reviews.apache.org/r/12164/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Virag Kothari
> 
>

[jira] [Commented] (OOZIE-1427) Update CredentialsModule docs to mention Hive

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-1427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13695713#comment-13695713
 ] 

Hadoop QA commented on OOZIE-1427:
--

Testing JIRA OOZIE-1427

Cleaning local svn workspace

> Update CredentialsModule docs to mention Hive
> -
>
> Key: OOZIE-1427
> URL: https://issues.apache.org/jira/browse/OOZIE-1427
> Project: Oozie
>  Issue Type: Bug
>  Components: docs
>Affects Versions: trunk
>Reporter: Robert Kanter
>Assignee: Robert Kanter
>Priority: Minor
> Fix For: trunk
>
> Attachments: OOZIE-1427.patch, OOZIE-1427.patch, OOZIE-1427.patch
>
>
> Update the CredentialsModule docs to make it clear that the HCatCredentails 
> can be used with Hive Metastore as well.  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (OOZIE-1436) Revert SLA_XML and few other varchar columns back to clob

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-1436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13695712#comment-13695712
 ] 

Hadoop QA commented on OOZIE-1436:
--

Testing JIRA OOZIE-1436

Cleaning local svn workspace



{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:red}-1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than 
132
.{color:red}-1{color} the patch does not add/modify any testcase
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT 
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc 
warnings
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac 
warnings
{color:red}-1 BACKWARDS_COMPATIBILITY{color}
.{color:red}-1{color} the patch seems to change 19 line(s) with JPA 
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:red}-1 TESTS{color}
.Tests run: 1252
.Tests failed: 0
.Tests errors: 1

.The patch failed the following testcases:

.  

{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch 


{color:red}*-1 Overall result, please check the reported -1(s)*{color}


The full output of the test-patch run is available at

.   https://builds.apache.org/job/oozie-trunk-precommit-build/631/

> Revert SLA_XML and few other varchar columns back to clob
> -
>
> Key: OOZIE-1436
> URL: https://issues.apache.org/jira/browse/OOZIE-1436
> Project: Oozie
>  Issue Type: Bug
>Reporter: Virag Kothari
>Assignee: Virag Kothari
> Fix For: trunk
>
> Attachments: OOZIE-1436.patch
>
>
> OOZIE-1398 converts few columns from clobs to varchar(4000). But there seems 
> to be problem where (@lenght=4000) annotation is not working and trying to 
> insert more than 1000 chars throws ORA-24816 with Oracle. Though it seems a 
> oracle problem from the error code, it is actually not as trying to do a 
> direct insert with more than 1000 chars in Oracle works. Also openjpa does 
> map to varchar2(4000) when creating schema. But somehow openjpa only accepts 
> upto 1000 chars and nothing more when trying to insert.
> Have posted on openjpa user list with no reply yet. Thread at 
> http://openjpa.208410.n2.nabble.com/ORA-24816-column-ordering-during-inserts-td7584233.html
> This jira will revert sla_xml and data varchar columns back to clob 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (OOZIE-1436) Revert SLA_XML and few other varchar columns back to clob

[Rohini Palaniswamy (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-1436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13695708#comment-13695708
 ] 

Rohini Palaniswamy commented on OOZIE-1436:
---

+1

> Revert SLA_XML and few other varchar columns back to clob
> -
>
> Key: OOZIE-1436
> URL: https://issues.apache.org/jira/browse/OOZIE-1436
> Project: Oozie
>  Issue Type: Bug
>Reporter: Virag Kothari
>Assignee: Virag Kothari
> Fix For: trunk
>
> Attachments: OOZIE-1436.patch
>
>
> OOZIE-1398 converts few columns from clobs to varchar(4000). But there seems 
> to be problem where (@lenght=4000) annotation is not working and trying to 
> insert more than 1000 chars throws ORA-24816 with Oracle. Though it seems a 
> oracle problem from the error code, it is actually not as trying to do a 
> direct insert with more than 1000 chars in Oracle works. Also openjpa does 
> map to varchar2(4000) when creating schema. But somehow openjpa only accepts 
> upto 1000 chars and nothing more when trying to insert.
> Have posted on openjpa user list with no reply yet. Thread at 
> http://openjpa.208410.n2.nabble.com/ORA-24816-column-ordering-during-inserts-td7584233.html
> This jira will revert sla_xml and data varchar columns back to clob 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (OOZIE-1439) Job materialization happening even after coordinator is supposed to be killed.

[Virag Kothari (JIRA)]

Virag Kothari created OOZIE-1439:


 Summary: Job materialization happening even after coordinator is 
supposed to be killed.
 Key: OOZIE-1439
 URL: https://issues.apache.org/jira/browse/OOZIE-1439
 Project: Oozie
  Issue Type: Bug
Reporter: Virag Kothari
Assignee: Virag Kothari


In some cases where executon of StatusTransitService and CoordKill are 
happening at same time, it may happen that the StatusTransitService overwrites 
the KILLED state of job to RUNNING 
This can be fixed by setting the doneMaterialization to true in CoordKillX
and failing the materializtion if the job's doneMaterialization to true. Also, 
eventually the StatusTransitService will convert the status back to KILLED 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Re: Review Request 12164: OOZIE-1436: Revert SLA_XML and few other varchar columns back to clob

[Rohini Palaniswamy]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/12164/#review22525
---

Ship it!


Ship It!

- Rohini Palaniswamy


On June 28, 2013, 5:03 p.m., Virag Kothari wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/12164/
> ---
> 
> (Updated June 28, 2013, 5:03 p.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-1436
> https://issues.apache.org/jira/browse/OOZIE-1436
> 
> 
> Repository: oozie
> 
> 
> Description
> ---
> 
> https://issues.apache.org/jira/browse/OOZIE-1436
> 
> Converting sla_xml column from wf_action, wf_job, coord_action, coord_job 
> tables back to clob. Also data column from wf_action converted back to clob
> 
> 
> Diffs
> -
> 
>   trunk/core/src/main/java/org/apache/oozie/CoordinatorActionBean.java 
> 1497646 
>   trunk/core/src/main/java/org/apache/oozie/CoordinatorJobBean.java 1497646 
>   trunk/core/src/main/java/org/apache/oozie/WorkflowActionBean.java 1497646 
>   trunk/core/src/main/java/org/apache/oozie/WorkflowJobBean.java 1497646 
>   
> trunk/core/src/main/java/org/apache/oozie/client/rest/JsonWorkflowAction.java 
> 1497646 
>   trunk/tools/src/main/java/org/apache/oozie/tools/OozieDBCLI.java 1497646 
> 
> Diff: https://reviews.apache.org/r/12164/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Virag Kothari
> 
>

[jira] [Commented] (OOZIE-1427) Update CredentialsModule docs to mention Hive

[Rohini Palaniswamy (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-1427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13695668#comment-13695668
 ] 

Rohini Palaniswamy commented on OOZIE-1427:
---

+1. Please go ahead with the commit.

> Update CredentialsModule docs to mention Hive
> -
>
> Key: OOZIE-1427
> URL: https://issues.apache.org/jira/browse/OOZIE-1427
> Project: Oozie
>  Issue Type: Bug
>  Components: docs
>Affects Versions: trunk
>Reporter: Robert Kanter
>Assignee: Robert Kanter
>Priority: Minor
> Fix For: trunk
>
> Attachments: OOZIE-1427.patch, OOZIE-1427.patch, OOZIE-1427.patch
>
>
> Update the CredentialsModule docs to make it clear that the HCatCredentails 
> can be used with Hive Metastore as well.  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (OOZIE-1427) Update CredentialsModule docs to mention Hive

[Robert Kanter (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-1427?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robert Kanter updated OOZIE-1427:
-

Attachment: OOZIE-1427.patch

Good catch.  I also missed .apache. earlier too.  

New patch fixes this.

> Update CredentialsModule docs to mention Hive
> -
>
> Key: OOZIE-1427
> URL: https://issues.apache.org/jira/browse/OOZIE-1427
> Project: Oozie
>  Issue Type: Bug
>  Components: docs
>Affects Versions: trunk
>Reporter: Robert Kanter
>Assignee: Robert Kanter
>Priority: Minor
> Fix For: trunk
>
> Attachments: OOZIE-1427.patch, OOZIE-1427.patch, OOZIE-1427.patch
>
>
> Update the CredentialsModule docs to make it clear that the HCatCredentails 
> can be used with Hive Metastore as well.  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (OOZIE-1433) ActionCheckX should override XCommand.getKey() to prevent duplicates

[Rohini Palaniswamy (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-1433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13695660#comment-13695660
 ] 

Rohini Palaniswamy commented on OOZIE-1433:
---

+1

> ActionCheckX should override XCommand.getKey() to prevent duplicates 
> -
>
> Key: OOZIE-1433
> URL: https://issues.apache.org/jira/browse/OOZIE-1433
> Project: Oozie
>  Issue Type: Bug
>Reporter: Virag Kothari
>Assignee: Virag Kothari
> Fix For: trunk
>
> Attachments: OOZIE-1433.patch, OOZIE-1433.patch
>
>
> ActionCheckX doesn't override XCommand.getKey(). Hence when 
> ActionCheckerService runs, ActionCheckX command for the same job is seen to 
> be duplicated in the queue.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (OOZIE-1427) Update CredentialsModule docs to mention Hive

[Rohini Palaniswamy (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-1427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13695658#comment-13695658
 ] 

Rohini Palaniswamy commented on OOZIE-1427:
---

{noformat}
+   1. HBase: =org.apache.oozie.action.hadoop.HBaseCredentials=

hcat=org.oozie.action.hadoop.HCatCredentials,
 hive=org.oozie.action.hadoop.HBaseCredentials
{noformat}
  Missed this one before. Should be 

{noformat}

+   1. HBase: =org.apache.oozie.action.hadoop.HbaseCredentials=
...
hcat=org.apache.oozie.action.hadoop.HCatCredentials,
hbase=org.apache.oozie.action.hadoop.HbaseCredentials
{noformat}

apache is missing in package name. And HBaseCredentials should be 
HbaseCredentials

> Update CredentialsModule docs to mention Hive
> -
>
> Key: OOZIE-1427
> URL: https://issues.apache.org/jira/browse/OOZIE-1427
> Project: Oozie
>  Issue Type: Bug
>  Components: docs
>Affects Versions: trunk
>Reporter: Robert Kanter
>Assignee: Robert Kanter
>Priority: Minor
> Fix For: trunk
>
> Attachments: OOZIE-1427.patch, OOZIE-1427.patch
>
>
> Update the CredentialsModule docs to make it clear that the HCatCredentails 
> can be used with Hive Metastore as well.  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Re: Branching for Oozie 4.0

[Robert Kanter]

Ah, you're right; thanks.

I reviewed the patch, but I'd like someone else to also take a look because
I'm not a DB expert.

- Robert


On Fri, Jun 28, 2013 at 11:30 AM, Virag Kothari  wrote:

> That change is reverted by OOZIE-1436. Can you review
> https://reviews.apache.org/r/12164/ and confirm?
> We can keep OOZIE-1437 open to deal the problems we will face when having
> a 'proper' fix for the clob/varchar issues.
>
> Thanks,
> Virag
>
> On 6/28/13 11:12 AM, "Robert Kanter"  wrote:
>
> >I think we need both; OOZIE-1436 reverts the columns back because of an
> >OpenJPA/Oracle issue and OOZIE-1437 is to fix an if statement added into
> >our code (see
> >
> https://issues.apache.org/jira/browse/OOZIE-1398?focusedCommentId=13694224
> >&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#com
> >ment-13694224
> >)
> >
> >thanks
> >- Robert
> >
> >
> >On Fri, Jun 28, 2013 at 11:00 AM, Virag Kothari 
> >wrote:
> >
> >> OOZIE-1436 will convert data varchar column back to lob due to some
> >>issues
> >> seen (In JIRA description) with OpenJPA.
> >>
> >>
> >> Thanks,
> >> Virag
> >>
> >> On 6/28/13 10:55 AM, "Robert Kanter"  wrote:
> >>
> >> >Can we also get OOZIE-1437 (paramerize size limit of setData in
> >> >JsonWorkflowAction based on user's configuration) in branch-4.0?
> >>Without
> >> >this, the size of  is always hardcoded at max of 2k,
> >>even
> >> >if the user changes it in oozie-site (because of OOZIE-1398).
> >> >
> >> >thanks
> >> >- Robert
> >> >
> >> >
> >> >On Fri, Jun 28, 2013 at 9:21 AM, Rohini Palaniswamy
> >> > >> >> wrote:
> >> >
> >> >> Hi,
> >> >> I am planning to branch trunk as branch-4.0 and trash the current
> >> >> branch-4.0 today evening. Discussed this with some of the folks
> >>during
> >> >>the
> >> >> summit. This will facilitate releasing HCat integration, JMS
> >> >>notifications
> >> >> for job and SLA and SLA monitoring as Oozie 4.0 which we are done
> >>with
> >> >> functional testing and in the process of stress testing. This will
> >>free
> >> >>up
> >> >> the trunk for the next release features of HA, cron based
> >>scheduling, DB
> >> >> clob column removal to move forward.
> >> >>
> >> >>
> >> >> Patches that need to get into branch-4.0 are
> >> >>
> >> >> OOZIE-1433 ActionCheckX should override XCommand.getKey() to prevent
> >> >> duplicates
> >> >> OOZIE-1436 Revert SLA_XML and few other varchar columns back to clob
> >> >> OOZIE-1427 Update CredentialsModule docs to mention Hive
> >> >> OOZIE-1379 Generate SLA end_miss event only after confirming against
> >> >> persistent store
> >> >> OOZIE-1435 StatusTransitService unnecessarily updates the
> >> >>lastModifiedTime
> >> >> of jobs which causes MaterializationService to bring same jobs in
> >>memory
> >> >> OOZIE-1249 and OOZIE-1430 Documentation for JMS notifications and SLA
> >> >> monitoring
> >> >> Subworkflow notifications don't have workflow as parent id
> >> >> And any other issues that crop up during the stress testing.
> >> >>
> >> >>Hoping most of this will get committed today before we branch.
> >>Please
> >> >> bring up any other jiras that we need to get into branch-4 before
> >>moving
> >> >> towards a release. Looking at a timeline of releasing 4.0 in 3 weeks.
> >> >>
> >> >> Regards,
> >> >> Rohini
> >> >>
> >>
> >>
>
>

Re: Review Request 12164: OOZIE-1436: Revert SLA_XML and few other varchar columns back to clob

[Robert Kanter]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/12164/#review22523
---


Looks good to me, but I'd like someone else to also take a look because I'm not 
a DB expert.

- Robert Kanter


On June 28, 2013, 5:03 p.m., Virag Kothari wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/12164/
> ---
> 
> (Updated June 28, 2013, 5:03 p.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-1436
> https://issues.apache.org/jira/browse/OOZIE-1436
> 
> 
> Repository: oozie
> 
> 
> Description
> ---
> 
> https://issues.apache.org/jira/browse/OOZIE-1436
> 
> Converting sla_xml column from wf_action, wf_job, coord_action, coord_job 
> tables back to clob. Also data column from wf_action converted back to clob
> 
> 
> Diffs
> -
> 
>   trunk/core/src/main/java/org/apache/oozie/CoordinatorActionBean.java 
> 1497646 
>   trunk/core/src/main/java/org/apache/oozie/CoordinatorJobBean.java 1497646 
>   trunk/core/src/main/java/org/apache/oozie/WorkflowActionBean.java 1497646 
>   trunk/core/src/main/java/org/apache/oozie/WorkflowJobBean.java 1497646 
>   
> trunk/core/src/main/java/org/apache/oozie/client/rest/JsonWorkflowAction.java 
> 1497646 
>   trunk/tools/src/main/java/org/apache/oozie/tools/OozieDBCLI.java 1497646 
> 
> Diff: https://reviews.apache.org/r/12164/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Virag Kothari
> 
>

[jira] [Commented] (OOZIE-1435) StatusTransitService unnecessarily updates the lastModifiedTime of jobs which causes MaterializationService to bring same jobs in memory

[Rohini Palaniswamy (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-1435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13695646#comment-13695646
 ] 

Rohini Palaniswamy commented on OOZIE-1435:
---

Apart from the 2 jenkins issues, please avoid assertNotNull(jpaService); in 
test code.  

> StatusTransitService unnecessarily updates the lastModifiedTime of jobs which 
> causes MaterializationService to bring same jobs in memory
> 
>
> Key: OOZIE-1435
> URL: https://issues.apache.org/jira/browse/OOZIE-1435
> Project: Oozie
>  Issue Type: Bug
>Reporter: Virag Kothari
>Assignee: Virag Kothari
> Fix For: trunk
>
> Attachments: OOZIE-1435.patch
>
>
> CoordMaterializationTriggerService
> brings 50 actions for materializing every 5 mins based on last modified
> time. It updates the last modified time every time for each of this action so
> new jobs can get a chance in the next interval. However, StatusTransitService
> unnecessarily keeps updating the last modified time which breaks the logic of
> materialization service as it may keep on bringing the same actions for
> materialization every scheduled interval.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Re: Branching for Oozie 4.0

[Virag Kothari]

That change is reverted by OOZIE-1436. Can you review
https://reviews.apache.org/r/12164/ and confirm?
We can keep OOZIE-1437 open to deal the problems we will face when having
a 'proper' fix for the clob/varchar issues.

Thanks,
Virag

On 6/28/13 11:12 AM, "Robert Kanter"  wrote:

>I think we need both; OOZIE-1436 reverts the columns back because of an
>OpenJPA/Oracle issue and OOZIE-1437 is to fix an if statement added into
>our code (see
>https://issues.apache.org/jira/browse/OOZIE-1398?focusedCommentId=13694224
>&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#com
>ment-13694224
>)
>
>thanks
>- Robert
>
>
>On Fri, Jun 28, 2013 at 11:00 AM, Virag Kothari 
>wrote:
>
>> OOZIE-1436 will convert data varchar column back to lob due to some
>>issues
>> seen (In JIRA description) with OpenJPA.
>>
>>
>> Thanks,
>> Virag
>>
>> On 6/28/13 10:55 AM, "Robert Kanter"  wrote:
>>
>> >Can we also get OOZIE-1437 (paramerize size limit of setData in
>> >JsonWorkflowAction based on user's configuration) in branch-4.0?
>>Without
>> >this, the size of  is always hardcoded at max of 2k,
>>even
>> >if the user changes it in oozie-site (because of OOZIE-1398).
>> >
>> >thanks
>> >- Robert
>> >
>> >
>> >On Fri, Jun 28, 2013 at 9:21 AM, Rohini Palaniswamy
>> >> >> wrote:
>> >
>> >> Hi,
>> >> I am planning to branch trunk as branch-4.0 and trash the current
>> >> branch-4.0 today evening. Discussed this with some of the folks
>>during
>> >>the
>> >> summit. This will facilitate releasing HCat integration, JMS
>> >>notifications
>> >> for job and SLA and SLA monitoring as Oozie 4.0 which we are done
>>with
>> >> functional testing and in the process of stress testing. This will
>>free
>> >>up
>> >> the trunk for the next release features of HA, cron based
>>scheduling, DB
>> >> clob column removal to move forward.
>> >>
>> >>
>> >> Patches that need to get into branch-4.0 are
>> >>
>> >> OOZIE-1433 ActionCheckX should override XCommand.getKey() to prevent
>> >> duplicates
>> >> OOZIE-1436 Revert SLA_XML and few other varchar columns back to clob
>> >> OOZIE-1427 Update CredentialsModule docs to mention Hive
>> >> OOZIE-1379 Generate SLA end_miss event only after confirming against
>> >> persistent store
>> >> OOZIE-1435 StatusTransitService unnecessarily updates the
>> >>lastModifiedTime
>> >> of jobs which causes MaterializationService to bring same jobs in
>>memory
>> >> OOZIE-1249 and OOZIE-1430 Documentation for JMS notifications and SLA
>> >> monitoring
>> >> Subworkflow notifications don't have workflow as parent id
>> >> And any other issues that crop up during the stress testing.
>> >>
>> >>Hoping most of this will get committed today before we branch.
>>Please
>> >> bring up any other jiras that we need to get into branch-4 before
>>moving
>> >> towards a release. Looking at a timeline of releasing 4.0 in 3 weeks.
>> >>
>> >> Regards,
>> >> Rohini
>> >>
>>
>>

[jira] [Created] (OOZIE-1438) parentID is null in job event message of subworkflow action

[Ryota Egashira (JIRA)]

Ryota Egashira created OOZIE-1438:
-

 Summary: parentID is null in job event message of subworkflow 
action 
 Key: OOZIE-1438
 URL: https://issues.apache.org/jira/browse/OOZIE-1438
 Project: Oozie
  Issue Type: Bug
Affects Versions: trunk
Reporter: Ryota Egashira
Assignee: Ryota Egashira




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (OOZIE-1433) ActionCheckX should override XCommand.getKey() to prevent duplicates

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-1433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13695638#comment-13695638
 ] 

Hadoop QA commented on OOZIE-1433:
--

Testing JIRA OOZIE-1433

Cleaning local svn workspace



{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:red}-1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than 
132
.{color:red}-1{color} the patch does not add/modify any testcase
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT 
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc 
warnings
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac 
warnings
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA 
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:red}-1 TESTS{color}
.Tests run: 1252
.Tests failed: 1
.Tests errors: 2

.The patch failed the following testcases:

.  
testEvictionOnTimeToIdle(org.apache.oozie.service.TestPartitionDependencyManagerEhcache)

{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch 


{color:red}*-1 Overall result, please check the reported -1(s)*{color}


The full output of the test-patch run is available at

.   https://builds.apache.org/job/oozie-trunk-precommit-build/630/

> ActionCheckX should override XCommand.getKey() to prevent duplicates 
> -
>
> Key: OOZIE-1433
> URL: https://issues.apache.org/jira/browse/OOZIE-1433
> Project: Oozie
>  Issue Type: Bug
>Reporter: Virag Kothari
>Assignee: Virag Kothari
> Fix For: trunk
>
> Attachments: OOZIE-1433.patch, OOZIE-1433.patch
>
>
> ActionCheckX doesn't override XCommand.getKey(). Hence when 
> ActionCheckerService runs, ActionCheckX command for the same job is seen to 
> be duplicated in the queue.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Re: Branching for Oozie 4.0

[Robert Kanter]

I think we need both; OOZIE-1436 reverts the columns back because of an
OpenJPA/Oracle issue and OOZIE-1437 is to fix an if statement added into
our code (see
https://issues.apache.org/jira/browse/OOZIE-1398?focusedCommentId=13694224&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13694224
)

thanks
- Robert


On Fri, Jun 28, 2013 at 11:00 AM, Virag Kothari  wrote:

> OOZIE-1436 will convert data varchar column back to lob due to some issues
> seen (In JIRA description) with OpenJPA.
>
>
> Thanks,
> Virag
>
> On 6/28/13 10:55 AM, "Robert Kanter"  wrote:
>
> >Can we also get OOZIE-1437 (paramerize size limit of setData in
> >JsonWorkflowAction based on user's configuration) in branch-4.0?  Without
> >this, the size of  is always hardcoded at max of 2k, even
> >if the user changes it in oozie-site (because of OOZIE-1398).
> >
> >thanks
> >- Robert
> >
> >
> >On Fri, Jun 28, 2013 at 9:21 AM, Rohini Palaniswamy
> > >> wrote:
> >
> >> Hi,
> >> I am planning to branch trunk as branch-4.0 and trash the current
> >> branch-4.0 today evening. Discussed this with some of the folks during
> >>the
> >> summit. This will facilitate releasing HCat integration, JMS
> >>notifications
> >> for job and SLA and SLA monitoring as Oozie 4.0 which we are done with
> >> functional testing and in the process of stress testing. This will free
> >>up
> >> the trunk for the next release features of HA, cron based scheduling, DB
> >> clob column removal to move forward.
> >>
> >>
> >> Patches that need to get into branch-4.0 are
> >>
> >> OOZIE-1433 ActionCheckX should override XCommand.getKey() to prevent
> >> duplicates
> >> OOZIE-1436 Revert SLA_XML and few other varchar columns back to clob
> >> OOZIE-1427 Update CredentialsModule docs to mention Hive
> >> OOZIE-1379 Generate SLA end_miss event only after confirming against
> >> persistent store
> >> OOZIE-1435 StatusTransitService unnecessarily updates the
> >>lastModifiedTime
> >> of jobs which causes MaterializationService to bring same jobs in memory
> >> OOZIE-1249 and OOZIE-1430 Documentation for JMS notifications and SLA
> >> monitoring
> >> Subworkflow notifications don't have workflow as parent id
> >> And any other issues that crop up during the stress testing.
> >>
> >>Hoping most of this will get committed today before we branch. Please
> >> bring up any other jiras that we need to get into branch-4 before moving
> >> towards a release. Looking at a timeline of releasing 4.0 in 3 weeks.
> >>
> >> Regards,
> >> Rohini
> >>
>
>

Re: Branching for Oozie 4.0

[Virag Kothari]

OOZIE-1436 will convert data varchar column back to lob due to some issues
seen (In JIRA description) with OpenJPA.


Thanks,
Virag

On 6/28/13 10:55 AM, "Robert Kanter"  wrote:

>Can we also get OOZIE-1437 (paramerize size limit of setData in
>JsonWorkflowAction based on user's configuration) in branch-4.0?  Without
>this, the size of  is always hardcoded at max of 2k, even
>if the user changes it in oozie-site (because of OOZIE-1398).
>
>thanks
>- Robert
>
>
>On Fri, Jun 28, 2013 at 9:21 AM, Rohini Palaniswamy
>> wrote:
>
>> Hi,
>> I am planning to branch trunk as branch-4.0 and trash the current
>> branch-4.0 today evening. Discussed this with some of the folks during
>>the
>> summit. This will facilitate releasing HCat integration, JMS
>>notifications
>> for job and SLA and SLA monitoring as Oozie 4.0 which we are done with
>> functional testing and in the process of stress testing. This will free
>>up
>> the trunk for the next release features of HA, cron based scheduling, DB
>> clob column removal to move forward.
>>
>>
>> Patches that need to get into branch-4.0 are
>>
>> OOZIE-1433 ActionCheckX should override XCommand.getKey() to prevent
>> duplicates
>> OOZIE-1436 Revert SLA_XML and few other varchar columns back to clob
>> OOZIE-1427 Update CredentialsModule docs to mention Hive
>> OOZIE-1379 Generate SLA end_miss event only after confirming against
>> persistent store
>> OOZIE-1435 StatusTransitService unnecessarily updates the
>>lastModifiedTime
>> of jobs which causes MaterializationService to bring same jobs in memory
>> OOZIE-1249 and OOZIE-1430 Documentation for JMS notifications and SLA
>> monitoring
>> Subworkflow notifications don't have workflow as parent id
>> And any other issues that crop up during the stress testing.
>>
>>Hoping most of this will get committed today before we branch. Please
>> bring up any other jiras that we need to get into branch-4 before moving
>> towards a release. Looking at a timeline of releasing 4.0 in 3 weeks.
>>
>> Regards,
>> Rohini
>>

Re: Branching for Oozie 4.0

[Robert Kanter]

Can we also get OOZIE-1437 (paramerize size limit of setData in
JsonWorkflowAction based on user's configuration) in branch-4.0?  Without
this, the size of  is always hardcoded at max of 2k, even
if the user changes it in oozie-site (because of OOZIE-1398).

thanks
- Robert


On Fri, Jun 28, 2013 at 9:21 AM, Rohini Palaniswamy  wrote:

> Hi,
> I am planning to branch trunk as branch-4.0 and trash the current
> branch-4.0 today evening. Discussed this with some of the folks during the
> summit. This will facilitate releasing HCat integration, JMS notifications
> for job and SLA and SLA monitoring as Oozie 4.0 which we are done with
> functional testing and in the process of stress testing. This will free up
> the trunk for the next release features of HA, cron based scheduling, DB
> clob column removal to move forward.
>
>
> Patches that need to get into branch-4.0 are
>
> OOZIE-1433 ActionCheckX should override XCommand.getKey() to prevent
> duplicates
> OOZIE-1436 Revert SLA_XML and few other varchar columns back to clob
> OOZIE-1427 Update CredentialsModule docs to mention Hive
> OOZIE-1379 Generate SLA end_miss event only after confirming against
> persistent store
> OOZIE-1435 StatusTransitService unnecessarily updates the lastModifiedTime
> of jobs which causes MaterializationService to bring same jobs in memory
> OOZIE-1249 and OOZIE-1430 Documentation for JMS notifications and SLA
> monitoring
> Subworkflow notifications don't have workflow as parent id
> And any other issues that crop up during the stress testing.
>
>Hoping most of this will get committed today before we branch. Please
> bring up any other jiras that we need to get into branch-4 before moving
> towards a release. Looking at a timeline of releasing 4.0 in 3 weeks.
>
> Regards,
> Rohini
>

[jira] [Updated] (OOZIE-1427) Update CredentialsModule docs to mention Hive

[Robert Kanter (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-1427?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robert Kanter updated OOZIE-1427:
-

Attachment: OOZIE-1427.patch

Updated patch based on Rohini's comments.

> Update CredentialsModule docs to mention Hive
> -
>
> Key: OOZIE-1427
> URL: https://issues.apache.org/jira/browse/OOZIE-1427
> Project: Oozie
>  Issue Type: Bug
>  Components: docs
>Affects Versions: trunk
>Reporter: Robert Kanter
>Assignee: Robert Kanter
>Priority: Minor
> Fix For: trunk
>
> Attachments: OOZIE-1427.patch, OOZIE-1427.patch
>
>
> Update the CredentialsModule docs to make it clear that the HCatCredentails 
> can be used with Hive Metastore as well.  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (OOZIE-1433) ActionCheckX should override XCommand.getKey() to prevent duplicates

[Virag Kothari (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-1433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Virag Kothari updated OOZIE-1433:
-

Attachment: OOZIE-1433.patch

> ActionCheckX should override XCommand.getKey() to prevent duplicates 
> -
>
> Key: OOZIE-1433
> URL: https://issues.apache.org/jira/browse/OOZIE-1433
> Project: Oozie
>  Issue Type: Bug
>Reporter: Virag Kothari
>Assignee: Virag Kothari
> Fix For: trunk
>
> Attachments: OOZIE-1433.patch, OOZIE-1433.patch
>
>
> ActionCheckX doesn't override XCommand.getKey(). Hence when 
> ActionCheckerService runs, ActionCheckX command for the same job is seen to 
> be duplicated in the queue.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (OOZIE-1436) Revert SLA_XML and few other varchar columns back to clob

[Virag Kothari (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-1436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Virag Kothari updated OOZIE-1436:
-

Attachment: OOZIE-1436.patch

Converting sla_xml column from wf_action, wf_job, coord_action, coord_job 
tables back to clob. Also data column from wf_action converted back to clob

> Revert SLA_XML and few other varchar columns back to clob
> -
>
> Key: OOZIE-1436
> URL: https://issues.apache.org/jira/browse/OOZIE-1436
> Project: Oozie
>  Issue Type: Bug
>Reporter: Virag Kothari
>Assignee: Virag Kothari
> Attachments: OOZIE-1436.patch
>
>
> OOZIE-1398 converts few columns from clobs to varchar(4000). But there seems 
> to be problem where (@lenght=4000) annotation is not working and trying to 
> insert more than 1000 chars throws ORA-24816 with Oracle. Though it seems a 
> oracle problem from the error code, it is actually not as trying to do a 
> direct insert with more than 1000 chars in Oracle works. Also openjpa does 
> map to varchar2(4000) when creating schema. But somehow openjpa only accepts 
> upto 1000 chars and nothing more when trying to insert.
> Have posted on openjpa user list with no reply yet. Thread at 
> http://openjpa.208410.n2.nabble.com/ORA-24816-column-ordering-during-inserts-td7584233.html
> This jira will revert sla_xml and data varchar columns back to clob 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Review Request 12164: OOZIE-1436: Revert SLA_XML and few other varchar columns back to clob

[Virag Kothari]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/12164/
---

Review request for oozie.


Bugs: OOZIE-1436
https://issues.apache.org/jira/browse/OOZIE-1436


Repository: oozie


Description
---

https://issues.apache.org/jira/browse/OOZIE-1436

Converting sla_xml column from wf_action, wf_job, coord_action, coord_job 
tables back to clob. Also data column from wf_action converted back to clob


Diffs
-

  trunk/core/src/main/java/org/apache/oozie/CoordinatorActionBean.java 1497646 
  trunk/core/src/main/java/org/apache/oozie/CoordinatorJobBean.java 1497646 
  trunk/core/src/main/java/org/apache/oozie/WorkflowActionBean.java 1497646 
  trunk/core/src/main/java/org/apache/oozie/WorkflowJobBean.java 1497646 
  trunk/core/src/main/java/org/apache/oozie/client/rest/JsonWorkflowAction.java 
1497646 
  trunk/tools/src/main/java/org/apache/oozie/tools/OozieDBCLI.java 1497646 

Diff: https://reviews.apache.org/r/12164/diff/


Testing
---


Thanks,

Virag Kothari

Branching for Oozie 4.0

[Rohini Palaniswamy]

Hi,
I am planning to branch trunk as branch-4.0 and trash the current
branch-4.0 today evening. Discussed this with some of the folks during the
summit. This will facilitate releasing HCat integration, JMS notifications
for job and SLA and SLA monitoring as Oozie 4.0 which we are done with
functional testing and in the process of stress testing. This will free up
the trunk for the next release features of HA, cron based scheduling, DB
clob column removal to move forward.


Patches that need to get into branch-4.0 are

OOZIE-1433 ActionCheckX should override XCommand.getKey() to prevent
duplicates
OOZIE-1436 Revert SLA_XML and few other varchar columns back to clob
OOZIE-1427 Update CredentialsModule docs to mention Hive
OOZIE-1379 Generate SLA end_miss event only after confirming against
persistent store
OOZIE-1435 StatusTransitService unnecessarily updates the lastModifiedTime
of jobs which causes MaterializationService to bring same jobs in memory
OOZIE-1249 and OOZIE-1430 Documentation for JMS notifications and SLA
monitoring
Subworkflow notifications don't have workflow as parent id
And any other issues that crop up during the stress testing.

   Hoping most of this will get committed today before we branch. Please
bring up any other jiras that we need to get into branch-4 before moving
towards a release. Looking at a timeline of releasing 4.0 in 3 weeks.

Regards,
Rohini

[jira] [Commented] (OOZIE-1433) ActionCheckX should override XCommand.getKey() to prevent duplicates

[Rohini Palaniswamy (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-1433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13695539#comment-13695539
 ] 

Rohini Palaniswamy commented on OOZIE-1433:
---

+1. Can you reupload the patch for jenkins to run again.

> ActionCheckX should override XCommand.getKey() to prevent duplicates 
> -
>
> Key: OOZIE-1433
> URL: https://issues.apache.org/jira/browse/OOZIE-1433
> Project: Oozie
>  Issue Type: Bug
>Reporter: Virag Kothari
>Assignee: Virag Kothari
> Fix For: trunk
>
> Attachments: OOZIE-1433.patch
>
>
> ActionCheckX doesn't override XCommand.getKey(). Hence when 
> ActionCheckerService runs, ActionCheckX command for the same job is seen to 
> be duplicated in the queue.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (OOZIE-1427) Update CredentialsModule docs to mention Hive

[Rohini Palaniswamy (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-1427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13695524#comment-13695524
 ] 

Rohini Palaniswamy commented on OOZIE-1427:
---

Few minor comments. 

bq. 1. HCat and Hive: =org.apache.oozie.action.hadoop.HCatCredentials=
  HCatalog and Hive Metastore

bq. *Note:* Despite being called HCatCredentails and using "hcat" in its 
property names, it will also work with Hive
  Can we change this to HCatalog Metastore and Hive Metastore are one and the 
same and the "hcat" type credential can be used with hive action to talk to a 
secure Hive Metastore. 

Can you remove the non-Javadoc comment while you are there. We are trying to 
remove it from code as much as possible when we modify classes. 

> Update CredentialsModule docs to mention Hive
> -
>
> Key: OOZIE-1427
> URL: https://issues.apache.org/jira/browse/OOZIE-1427
> Project: Oozie
>  Issue Type: Bug
>  Components: docs
>Affects Versions: trunk
>Reporter: Robert Kanter
>Assignee: Robert Kanter
>Priority: Minor
> Fix For: trunk
>
> Attachments: OOZIE-1427.patch
>
>
> Update the CredentialsModule docs to make it clear that the HCatCredentails 
> can be used with Hive Metastore as well.  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (OOZIE-1398) [Scale] Reduce the number of CLOB columns used

[Rohini Palaniswamy (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-1398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13695537#comment-13695537
 ] 

Rohini Palaniswamy commented on OOZIE-1398:
---

Yes. We will be converting all CLOB columns to VARBINARY (can store up to 4MB) 
in next release and compress and store data in there. postgres bytea is 2G.  
VARBINARY in mysql has lesser limit though - max row size of 65K, but with 
compression that should be good. For 4.0, we will revert back the sla_xml and 
data columns back to CLOB due to openjpa issue with Oracle and ordering of 
columns during insert (OOZIE-1436). We will only keep error message column as 
varchar in 4.0.  

> [Scale] Reduce the number of CLOB columns used
> --
>
> Key: OOZIE-1398
> URL: https://issues.apache.org/jira/browse/OOZIE-1398
> Project: Oozie
>  Issue Type: Improvement
>Affects Versions: trunk, 3.3.2
>Reporter: Rohini Palaniswamy
>Assignee: Ryota Egashira
> Fix For: trunk
>
> Attachments: OOZIE-1398-v4.patch, OOZIE-1398-v5.patch, 
> OOZIE-1398-v7.patch, OOZIE-1398-v8.patch, OOZIE-1398-v8.patch
>
>
>   When the number of concurrent submissions on Oozie increased to 100-200 per 
> minute, it was not able to scale and we hit Oracle issues as there were lot 
> of CLOB columns and DB became a bottle neck.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira