[jira] [Commented] (OOZIE-2436) Fork/join workflow fails with "oozie.action.yarn.tag must not be null"

[abhishek bafna (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15101078#comment-15101078
 ] 

abhishek bafna commented on OOZIE-2436:
---

Can you add more information about the issue?

> Fork/join workflow fails with "oozie.action.yarn.tag must not be null"
> --
>
> Key: OOZIE-2436
> URL: https://issues.apache.org/jira/browse/OOZIE-2436
> Project: Oozie
>  Issue Type: Bug
>Reporter: Purshotam Shah
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OOZIE-2751) LocalOozieClient is missing methods from OozieClient

[Abhishek Bafna (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-2751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Abhishek Bafna updated OOZIE-2751:
--
Attachment: OOZIE-2751-09.patch

> LocalOozieClient is missing methods from OozieClient
> 
>
> Key: OOZIE-2751
> URL: https://issues.apache.org/jira/browse/OOZIE-2751
> Project: Oozie
>  Issue Type: Bug
>Reporter: Robert Kanter
>Assignee: Abhishek Bafna
> Fix For: 5.0.0
>
> Attachments: OOZIE-2751-00.patch, OOZIE-2751-01.patch, 
> OOZIE-2751-02.patch, OOZIE-2751-03.patch, OOZIE-2751-04.patch, 
> OOZIE-2751-05.patch, OOZIE-2751-06.patch, OOZIE-2751-07.patch, 
> OOZIE-2751-08.patch, OOZIE-2751-09.patch
>
>
> {{LocalOozieClient}} works by wrapping direct calls to {{DagEngine}} in the 
> normal {{OozieClient}} methods instead of making REST calls to an Oozie 
> Server.  However, it currently doesn't override all methods, which means you 
> hit the {{OozieClient}} implementation, which tries to make a REST call and 
> will fail (because it's not setup for that).  For example, {{getStatus(String 
> jobId)}} isn't overridden, and doesn't work when used from 
> {{LocalOozieClient}}.
> We should also create a test that verifies that all methods are overridden.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (OOZIE-2751) LocalOozieClient is missing methods from OozieClient

[Abhishek Bafna (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-2751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Abhishek Bafna updated OOZIE-2751:
--
Attachment: OOZIE-2751-10.patch

[~rkanter] Addressing your review comments. Thanks.

> LocalOozieClient is missing methods from OozieClient
> 
>
> Key: OOZIE-2751
> URL: https://issues.apache.org/jira/browse/OOZIE-2751
> Project: Oozie
>  Issue Type: Bug
>Reporter: Robert Kanter
>Assignee: Abhishek Bafna
> Fix For: 5.0.0
>
> Attachments: OOZIE-2751-00.patch, OOZIE-2751-01.patch, 
> OOZIE-2751-02.patch, OOZIE-2751-03.patch, OOZIE-2751-04.patch, 
> OOZIE-2751-05.patch, OOZIE-2751-06.patch, OOZIE-2751-07.patch, 
> OOZIE-2751-08.patch, OOZIE-2751-09.patch, OOZIE-2751-10.patch
>
>
> {{LocalOozieClient}} works by wrapping direct calls to {{DagEngine}} in the 
> normal {{OozieClient}} methods instead of making REST calls to an Oozie 
> Server.  However, it currently doesn't override all methods, which means you 
> hit the {{OozieClient}} implementation, which tries to make a REST call and 
> will fail (because it's not setup for that).  For example, {{getStatus(String 
> jobId)}} isn't overridden, and doesn't work when used from 
> {{LocalOozieClient}}.
> We should also create a test that verifies that all methods are overridden.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (OOZIE-2751) LocalOozieClient is missing methods from OozieClient

[Abhishek Bafna (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16001875#comment-16001875
 ] 

Abhishek Bafna commented on OOZIE-2751:
---

Thanks [~rkanter] and others for reviews.

> LocalOozieClient is missing methods from OozieClient
> 
>
> Key: OOZIE-2751
> URL: https://issues.apache.org/jira/browse/OOZIE-2751
> Project: Oozie
>  Issue Type: Bug
>Reporter: Robert Kanter
>Assignee: Abhishek Bafna
> Fix For: 5.0.0
>
> Attachments: OOZIE-2751-00.patch, OOZIE-2751-01.patch, 
> OOZIE-2751-02.patch, OOZIE-2751-03.patch, OOZIE-2751-04.patch, 
> OOZIE-2751-05.patch, OOZIE-2751-06.patch, OOZIE-2751-07.patch, 
> OOZIE-2751-08.patch, OOZIE-2751-09.patch, OOZIE-2751-10.patch
>
>
> {{LocalOozieClient}} works by wrapping direct calls to {{DagEngine}} in the 
> normal {{OozieClient}} methods instead of making REST calls to an Oozie 
> Server.  However, it currently doesn't override all methods, which means you 
> hit the {{OozieClient}} implementation, which tries to make a REST call and 
> will fail (because it's not setup for that).  For example, {{getStatus(String 
> jobId)}} isn't overridden, and doesn't work when used from 
> {{LocalOozieClient}}.
> We should also create a test that verifies that all methods are overridden.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (OOZIE-2927) Append new line character for Hive2 query using query tag

[Abhishek Bafna (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-2927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Abhishek Bafna updated OOZIE-2927:
--
Fix Version/s: 5.0.0

> Append new line character for Hive2 query using query tag
> -
>
> Key: OOZIE-2927
> URL: https://issues.apache.org/jira/browse/OOZIE-2927
> Project: Oozie
>  Issue Type: Bug
>  Components: core
>Reporter: Abhishek Bafna
>Assignee: Abhishek Bafna
> Fix For: 5.0.0
>
>
> Without the new line character in the query, beeline does not do anything. 
> Though hive2 action completes successfully but query is not executed.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (OOZIE-2927) Append new line character for Hive2 query using query tag

[Abhishek Bafna (JIRA)]

Abhishek Bafna created OOZIE-2927:
-

 Summary: Append new line character for Hive2 query using query tag
 Key: OOZIE-2927
 URL: https://issues.apache.org/jira/browse/OOZIE-2927
 Project: Oozie
  Issue Type: Bug
  Components: core
Reporter: Abhishek Bafna
Assignee: Abhishek Bafna


Without the new line character in the query, beeline does not do anything. 
Though hive2 action completes successfully but query is not executed.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (OOZIE-2927) Append new line character for Hive2 query using query tag

[Abhishek Bafna (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-2927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Abhishek Bafna updated OOZIE-2927:
--
Attachment: OOZIE-2927-00.patch

> Append new line character for Hive2 query using query tag
> -
>
> Key: OOZIE-2927
> URL: https://issues.apache.org/jira/browse/OOZIE-2927
> Project: Oozie
>  Issue Type: Bug
>  Components: core
>Reporter: Abhishek Bafna
>Assignee: Abhishek Bafna
> Fix For: 5.0.0
>
> Attachments: OOZIE-2927-00.patch
>
>
> Without the new line character in the query, beeline does not do anything. 
> Though hive2 action completes successfully but query is not executed.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (OOZIE-2927) Append new line character for Hive2 query using query tag

[Abhishek Bafna (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-2927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Abhishek Bafna updated OOZIE-2927:
--
Attachment: OOZIE-2927-01.patch

> Append new line character for Hive2 query using query tag
> -
>
> Key: OOZIE-2927
> URL: https://issues.apache.org/jira/browse/OOZIE-2927
> Project: Oozie
>  Issue Type: Bug
>  Components: core
>Reporter: Abhishek Bafna
>Assignee: Abhishek Bafna
> Fix For: 5.0.0
>
> Attachments: OOZIE-2927-00.patch, OOZIE-2927-01.patch
>
>
> Without the new line character in the query, beeline does not do anything. 
> Though hive2 action completes successfully but query is not executed.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (OOZIE-2927) Append new line character for Hive2 query using query tag

[Abhishek Bafna (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16049051#comment-16049051
 ] 

Abhishek Bafna commented on OOZIE-2927:
---

Thanks [~gezapeti]. I think, I will modify the existing test case to cover the 
scenario.

> Append new line character for Hive2 query using query tag
> -
>
> Key: OOZIE-2927
> URL: https://issues.apache.org/jira/browse/OOZIE-2927
> Project: Oozie
>  Issue Type: Bug
>  Components: core
>Reporter: Abhishek Bafna
>Assignee: Abhishek Bafna
> Fix For: 5.0.0
>
> Attachments: OOZIE-2927-00.patch, OOZIE-2927-01.patch
>
>
> Without the new line character in the query, beeline does not do anything. 
> Though hive2 action completes successfully but query is not executed.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Resolved] (OOZIE-3080) testOldFormatRead fails with classnotfound

[Abhishek Bafna (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-3080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Abhishek Bafna resolved OOZIE-3080.
---
Resolution: Not A Problem

> testOldFormatRead fails with classnotfound
> --
>
> Key: OOZIE-3080
> URL: https://issues.apache.org/jira/browse/OOZIE-3080
> Project: Oozie
>  Issue Type: Bug
>  Components: tests
>Reporter: Yesha Vora
>
> testOldFormatRead UT fails with classnotfound
> {code}
> Error Message
> java.lang.ClassNotFoundException: null
> Stacktrace
> java.io.IOException: java.lang.ClassNotFoundException: null
>   at 
> org.apache.oozie.workflow.lite.NodeDef.readVersionZero(NodeDef.java:140)
>   at org.apache.oozie.workflow.lite.NodeDef.readFields(NodeDef.java:190)
>   at 
> org.apache.oozie.workflow.lite.LiteWorkflowApp.readFields(LiteWorkflowApp.java:147)
>   at 
> org.apache.oozie.workflow.lite.TestLiteWorkflowApp.testOldFormatRead(TestLiteWorkflowApp.java:75)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:498)
>   at junit.framework.TestCase.runTest(TestCase.java:168)
>   at junit.framework.TestCase.runBare(TestCase.java:134)
>   at junit.framework.TestResult$1.protect(TestResult.java:110)
>   at junit.framework.TestResult.runProtected(TestResult.java:128)
>   at junit.framework.TestResult.run(TestResult.java:113)
>   at junit.framework.TestCase.run(TestCase.java:124)
>   at junit.framework.TestSuite.runTest(TestSuite.java:243)
>   at junit.framework.TestSuite.run(TestSuite.java:238)
>   at 
> org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:83)
>   at org.junit.runners.Suite.runChild(Suite.java:128)
>   at org.junit.runners.Suite.runChild(Suite.java:24)
>   at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>   at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>   at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.ClassNotFoundException: null
>   at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
>   at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
>   at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
>   at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
>   at java.lang.Class.forName0(Native Method)
>   at java.lang.Class.forName(Class.java:264)
>   at 
> org.apache.oozie.workflow.lite.NodeDef.readVersionZero(NodeDef.java:137)
>   ... 24 more{code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (OOZIE-3139) Oozie validates workflow incorrectly

[Abhishek Bafna (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-3139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16279670#comment-16279670
 ] 

Abhishek Bafna commented on OOZIE-3139:
---

[~dionusos] Looks good to me. +1.
Can we also include the below scenario. Thanks.


{code}




































kill



{code}


> Oozie validates workflow incorrectly
> 
>
> Key: OOZIE-3139
> URL: https://issues.apache.org/jira/browse/OOZIE-3139
> Project: Oozie
>  Issue Type: Bug
>  Components: core
>Affects Versions: 4.3.0
>Reporter: Denes Bodo
>Assignee: Denes Bodo
>Priority: Critical
> Attachments: OOZIE-3139.001.patch, OOZIE-3139.002.patch, 
> exception.log, workflow.pdf
>
>
> We found that after OOZIE-1978 the attached workflow becomes invalid. Aa we 
> found out the following line brings the problem: 
> https://github.com/apache/oozie/commit/8e9b9042b3270dc5ff975c44a5c977fcc41250e4#diff-a38ae78886b7c3afb711d32cfb4dbbbdR267
>  . The parameter "okPath" changed from okPath to true.
> The following workflow is problematic:
> {noformat}
> 
> 
> 
> 
> 
> 
> 
> 
> ${jobTracker}
> ${nameNode}
> ${myscript}
> ${myscriptPath}
> 
> 
> 
> 
> 
> 
> 
> ${jobTracker}
> ${nameNode}
> ${myscript}
> ${myscriptPath}
> 
> 
> 
> 
> 
> 
> 
> 
> ${jobTracker}
> ${nameNode}
> ${myscript}
> ${myscriptPath}
> 
> 
> 
> 
> 
> 
> 
> ${jobTracker}
> ${nameNode}
> ${myscript}
> ${myscriptPath}
> 
> 
> 
> 
> 
> 
> 
> ${jobTracker}
> ${nameNode}
> ${myscript}
> ${myscriptPath}
> 
> 
> 
> 
>  
>
> 
> Shell action failed, error 
> message[${wf:errorMessage(wf:lastErrorNode())}]
> 
> 
> 
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (OOZIE-2582) Populating external child Ids for action failures

[Abhishek Bafna (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-2582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Abhishek Bafna updated OOZIE-2582:
--
Attachment: OOZIE-2582-07.patch

> Populating external child Ids for action failures
> -
>
> Key: OOZIE-2582
> URL: https://issues.apache.org/jira/browse/OOZIE-2582
> Project: Oozie
>  Issue Type: Bug
>  Components: core
>Reporter: Abhishek Bafna
>Assignee: Abhishek Bafna
> Fix For: 4.3.0
>
> Attachments: OOZIE-2582-00.patch, OOZIE-2582-01.patch, 
> OOZIE-2582-02.patch, OOZIE-2582-03.patch, OOZIE-2582-04.patch, 
> OOZIE-2582-05.patch, OOZIE-2582-06.patch, OOZIE-2582-07.patch
>
>
> Currently Oozie external child ids are populated into workflow bean, when the 
> job/action completes successfully. It should populate external child ids in 
> case of job failures as well.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2613) Upgrade hive version from 0.13.1 to 1.2.0

[Abhishek Bafna (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15517079#comment-15517079
 ] 

Abhishek Bafna commented on OOZIE-2613:
---

[~rkanter] and [~rohini] Does this looks good from Oozie 4.3.0 perspective. 
Thanks.

> Upgrade hive version from 0.13.1 to 1.2.0
> -
>
> Key: OOZIE-2613
> URL: https://issues.apache.org/jira/browse/OOZIE-2613
> Project: Oozie
>  Issue Type: Bug
>  Components: core
>Reporter: Abhishek Bafna
>Assignee: Abhishek Bafna
>Priority: Blocker
> Fix For: 4.3.0
>
> Attachments: OOZIE-2613-00.patch, OOZIE-2613-01.patch, 
> OOZIE-2613-02.patch, OOZIE-2613-03.patch, OOZIE-2613-05.patch, 
> OOZIE-2613-06.patch, OOZIE-2613-07.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2582) Populating external child Ids for action failures

[Abhishek Bafna (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15517093#comment-15517093
 ] 

Abhishek Bafna commented on OOZIE-2582:
---

The Hive2 testcase failed with {{default}} database not found. I guess I need 
to move the new negative testcase into the existing {{testHive2Action}} similar 
to one which it already have.

{{TestCoordActionsKillXCommand}} testcases passes locally.

> Populating external child Ids for action failures
> -
>
> Key: OOZIE-2582
> URL: https://issues.apache.org/jira/browse/OOZIE-2582
> Project: Oozie
>  Issue Type: Bug
>  Components: core
>Reporter: Abhishek Bafna
>Assignee: Abhishek Bafna
> Fix For: 4.3.0
>
> Attachments: OOZIE-2582-00.patch, OOZIE-2582-01.patch, 
> OOZIE-2582-02.patch, OOZIE-2582-03.patch, OOZIE-2582-04.patch, 
> OOZIE-2582-05.patch, OOZIE-2582-06.patch, OOZIE-2582-07.patch
>
>
> Currently Oozie external child ids are populated into workflow bean, when the 
> job/action completes successfully. It should populate external child ids in 
> case of job failures as well.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OOZIE-2582) Populating external child Ids for action failures

[Abhishek Bafna (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-2582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Abhishek Bafna updated OOZIE-2582:
--
Attachment: OOZIE-2582-08.patch

> Populating external child Ids for action failures
> -
>
> Key: OOZIE-2582
> URL: https://issues.apache.org/jira/browse/OOZIE-2582
> Project: Oozie
>  Issue Type: Bug
>  Components: core
>Reporter: Abhishek Bafna
>Assignee: Abhishek Bafna
> Fix For: 4.3.0
>
> Attachments: OOZIE-2582-00.patch, OOZIE-2582-01.patch, 
> OOZIE-2582-02.patch, OOZIE-2582-03.patch, OOZIE-2582-04.patch, 
> OOZIE-2582-05.patch, OOZIE-2582-06.patch, OOZIE-2582-07.patch, 
> OOZIE-2582-08.patch
>
>
> Currently Oozie external child ids are populated into workflow bean, when the 
> job/action completes successfully. It should populate external child ids in 
> case of job failures as well.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2582) Populating external child Ids for action failures

[Abhishek Bafna (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15517104#comment-15517104
 ] 

Abhishek Bafna commented on OOZIE-2582:
---

[~rohini], [~rkanter] The external child id population does not work for Hive2 
action. Hive2 uses the beeline client and submits the query to HiveServer2 for 
execution and beeline does not get any information about the launched jobs. 
Please correct me, If my understanding is wrong. Thanks.

> Populating external child Ids for action failures
> -
>
> Key: OOZIE-2582
> URL: https://issues.apache.org/jira/browse/OOZIE-2582
> Project: Oozie
>  Issue Type: Bug
>  Components: core
>Reporter: Abhishek Bafna
>Assignee: Abhishek Bafna
> Fix For: 4.3.0
>
> Attachments: OOZIE-2582-00.patch, OOZIE-2582-01.patch, 
> OOZIE-2582-02.patch, OOZIE-2582-03.patch, OOZIE-2582-04.patch, 
> OOZIE-2582-05.patch, OOZIE-2582-06.patch, OOZIE-2582-07.patch, 
> OOZIE-2582-08.patch
>
>
> Currently Oozie external child ids are populated into workflow bean, when the 
> job/action completes successfully. It should populate external child ids in 
> case of job failures as well.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2582) Populating external child Ids for action failures

[Abhishek Bafna (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15517256#comment-15517256
 ] 

Abhishek Bafna commented on OOZIE-2582:
---

[~rkanter] Does this means we will bump the priority for OOZIE-2613. Thanks. :)

> Populating external child Ids for action failures
> -
>
> Key: OOZIE-2582
> URL: https://issues.apache.org/jira/browse/OOZIE-2582
> Project: Oozie
>  Issue Type: Bug
>  Components: core
>Reporter: Abhishek Bafna
>Assignee: Abhishek Bafna
> Fix For: 4.3.0
>
> Attachments: OOZIE-2582-00.patch, OOZIE-2582-01.patch, 
> OOZIE-2582-02.patch, OOZIE-2582-03.patch, OOZIE-2582-04.patch, 
> OOZIE-2582-05.patch, OOZIE-2582-06.patch, OOZIE-2582-07.patch, 
> OOZIE-2582-08.patch
>
>
> Currently Oozie external child ids are populated into workflow bean, when the 
> job/action completes successfully. It should populate external child ids in 
> case of job failures as well.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (OOZIE-2582) Populating external child Ids for action failures

[Abhishek Bafna (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15517256#comment-15517256
 ] 

Abhishek Bafna edited comment on OOZIE-2582 at 9/23/16 6:57 PM:


[~rkanter] Does this means we will bump the priority for OOZIE-2613. Thanks. :)

Though patch does not remove any of the existing code from {{Hive2Main}}, so 
should be fine for others who use higher version of the {{hive}} and should 
continue to populate external child ids.


was (Author: abhishekbafna):
[~rkanter] Does this means we will bump the priority for OOZIE-2613. Thanks. :)

> Populating external child Ids for action failures
> -
>
> Key: OOZIE-2582
> URL: https://issues.apache.org/jira/browse/OOZIE-2582
> Project: Oozie
>  Issue Type: Bug
>  Components: core
>Reporter: Abhishek Bafna
>Assignee: Abhishek Bafna
> Fix For: 4.3.0
>
> Attachments: OOZIE-2582-00.patch, OOZIE-2582-01.patch, 
> OOZIE-2582-02.patch, OOZIE-2582-03.patch, OOZIE-2582-04.patch, 
> OOZIE-2582-05.patch, OOZIE-2582-06.patch, OOZIE-2582-07.patch, 
> OOZIE-2582-08.patch
>
>
> Currently Oozie external child ids are populated into workflow bean, when the 
> job/action completes successfully. It should populate external child ids in 
> case of job failures as well.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OOZIE-2041) Add an admin command to run the PurgeXCommand

[Abhishek Bafna (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-2041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Abhishek Bafna updated OOZIE-2041:
--
Attachment: OOZIE-2041-03.patch

> Add an admin command to run the PurgeXCommand
> -
>
> Key: OOZIE-2041
> URL: https://issues.apache.org/jira/browse/OOZIE-2041
> Project: Oozie
>  Issue Type: New Feature
>  Components: core
>Affects Versions: trunk
>Reporter: Robert Kanter
>Assignee: Abhishek Bafna
> Attachments: OOZIE-2041-00.patch, OOZIE-2041-01.patch, 
> OOZIE-2041-02.patch, OOZIE-2041-03.patch
>
>
> Some users may find it useful to be able to run the PurgeXCommand on-demand.  
> We can add a new admin command to run this.  e.g.
> {noformat}
> oozie admin -purge
> {noformat}
> With no args, it can use the "older than" values from oozie-site, but we 
> could have it take some arguments to override those for the command.
> Another thing to worry about is making sure that we don't run this if either 
> it's already running (i.e. the user sent it twice) or if the scheduled 
> PurgeService is already running it.  On top of that, we may need extra 
> consideration for HA setups where we currently only run it on the leader.  
> We should probably also have a to not schedule the PurgeService in ooize-site 
> for users who only want to run it manually.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2041) Add an admin command to run the PurgeXCommand

[Abhishek Bafna (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15517417#comment-15517417
 ] 

Abhishek Bafna commented on OOZIE-2041:
---

Yes [~rkanter]. The first option should work out of the box in a Oozie HA 
setup. As we configure ZK related extras services like {{ZKLocksService}}. 
{{ZKLocksService}} extends {{MemoryLocksService}} and only 
{{MemoryLocksService}} have the implementation for {{Class 
getInterface();}}. Removed the extra leader check. Also, added the 
documentation for REST API. Thanks for your feedback.

> Add an admin command to run the PurgeXCommand
> -
>
> Key: OOZIE-2041
> URL: https://issues.apache.org/jira/browse/OOZIE-2041
> Project: Oozie
>  Issue Type: New Feature
>  Components: core
>Affects Versions: trunk
>Reporter: Robert Kanter
>Assignee: Abhishek Bafna
> Attachments: OOZIE-2041-00.patch, OOZIE-2041-01.patch, 
> OOZIE-2041-02.patch, OOZIE-2041-03.patch
>
>
> Some users may find it useful to be able to run the PurgeXCommand on-demand.  
> We can add a new admin command to run this.  e.g.
> {noformat}
> oozie admin -purge
> {noformat}
> With no args, it can use the "older than" values from oozie-site, but we 
> could have it take some arguments to override those for the command.
> Another thing to worry about is making sure that we don't run this if either 
> it's already running (i.e. the user sent it twice) or if the scheduled 
> PurgeService is already running it.  On top of that, we may need extra 
> consideration for HA setups where we currently only run it on the leader.  
> We should probably also have a to not schedule the PurgeService in ooize-site 
> for users who only want to run it manually.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OOZIE-2582) Populating external child Ids for action failures

[Abhishek Bafna (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-2582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Abhishek Bafna updated OOZIE-2582:
--
Attachment: OOZIE-2582-09.patch

> Populating external child Ids for action failures
> -
>
> Key: OOZIE-2582
> URL: https://issues.apache.org/jira/browse/OOZIE-2582
> Project: Oozie
>  Issue Type: Bug
>  Components: core
>Reporter: Abhishek Bafna
>Assignee: Abhishek Bafna
> Fix For: 4.3.0
>
> Attachments: OOZIE-2582-00.patch, OOZIE-2582-01.patch, 
> OOZIE-2582-02.patch, OOZIE-2582-03.patch, OOZIE-2582-04.patch, 
> OOZIE-2582-05.patch, OOZIE-2582-06.patch, OOZIE-2582-07.patch, 
> OOZIE-2582-08.patch, OOZIE-2582-09.patch
>
>
> Currently Oozie external child ids are populated into workflow bean, when the 
> job/action completes successfully. It should populate external child ids in 
> case of job failures as well.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2582) Populating external child Ids for action failures

[Abhishek Bafna (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15519321#comment-15519321
 ] 

Abhishek Bafna commented on OOZIE-2582:
---

It should work for failure cases, as it is calling the 
{{writeExternalChildIDs}} in the {{writeExternalChildIDs}} block. I have added 
the {{Pattern}} for Hadoop 2 as well and added comment for external child id 
check, currently they are disabled. Thanks.

> Populating external child Ids for action failures
> -
>
> Key: OOZIE-2582
> URL: https://issues.apache.org/jira/browse/OOZIE-2582
> Project: Oozie
>  Issue Type: Bug
>  Components: core
>Reporter: Abhishek Bafna
>Assignee: Abhishek Bafna
> Fix For: 4.3.0
>
> Attachments: OOZIE-2582-00.patch, OOZIE-2582-01.patch, 
> OOZIE-2582-02.patch, OOZIE-2582-03.patch, OOZIE-2582-04.patch, 
> OOZIE-2582-05.patch, OOZIE-2582-06.patch, OOZIE-2582-07.patch, 
> OOZIE-2582-08.patch, OOZIE-2582-09.patch
>
>
> Currently Oozie external child ids are populated into workflow bean, when the 
> job/action completes successfully. It should populate external child ids in 
> case of job failures as well.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (OOZIE-2194) oozie job -kill doesn't work with spark action

[Abhishek Bafna (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-2194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Abhishek Bafna reassigned OOZIE-2194:
-

Assignee: Abhishek Bafna

> oozie job -kill doesn't work with spark action
> --
>
> Key: OOZIE-2194
> URL: https://issues.apache.org/jira/browse/OOZIE-2194
> Project: Oozie
>  Issue Type: Bug
>Reporter: Thomas Graves
>Assignee: Abhishek Bafna
>
> I was trying the spark action out with oozie and was able to launch fine. But 
> when I tried do use oozie job -kill it only killed the launcher job and not 
> the spark application itself.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OOZIE-2194) oozie job -kill doesn't work with spark action

[Abhishek Bafna (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-2194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Abhishek Bafna updated OOZIE-2194:
--
Attachment: OOZIE-2194-00.patch

> oozie job -kill doesn't work with spark action
> --
>
> Key: OOZIE-2194
> URL: https://issues.apache.org/jira/browse/OOZIE-2194
> Project: Oozie
>  Issue Type: Bug
>Reporter: Thomas Graves
>Assignee: Abhishek Bafna
> Fix For: 4.3.0
>
> Attachments: OOZIE-2194-00.patch
>
>
> I was trying the spark action out with oozie and was able to launch fine. But 
> when I tried do use oozie job -kill it only killed the launcher job and not 
> the spark application itself.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2678) Oozie job -kill doesn't work with tez jobs

[Abhishek Bafna (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15527163#comment-15527163
 ] 

Abhishek Bafna commented on OOZIE-2678:
---

Thanks [~rohini] for the review and committing it.

> Oozie job -kill doesn't work with tez jobs
> --
>
> Key: OOZIE-2678
> URL: https://issues.apache.org/jira/browse/OOZIE-2678
> Project: Oozie
>  Issue Type: Bug
>Reporter: Rohini Palaniswamy
>Assignee: Abhishek Bafna
> Fix For: 4.3.0
>
> Attachments: OOZIE-2678-00.patch, OOZIE-2678-01.patch, 
> OOZIE-2678-02.patch, OOZIE-2678-03.patch
>
>
>  OOZIE-2243 added support for mapreduce jobs. We need to do that for Tez jobs 
> as well. TEZ-909 added tez.application.tags similar to mapreduce.job.tags,



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2582) Populating external child Ids for action failures

[Abhishek Bafna (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15527170#comment-15527170
 ] 

Abhishek Bafna commented on OOZIE-2582:
---

Thanks [~rohini] and [~rkanter] for the reviews and committing it.

> Populating external child Ids for action failures
> -
>
> Key: OOZIE-2582
> URL: https://issues.apache.org/jira/browse/OOZIE-2582
> Project: Oozie
>  Issue Type: Bug
>  Components: core
>Reporter: Abhishek Bafna
>Assignee: Abhishek Bafna
> Fix For: 4.3.0
>
> Attachments: OOZIE-2582-00.patch, OOZIE-2582-01.patch, 
> OOZIE-2582-02.patch, OOZIE-2582-03.patch, OOZIE-2582-04.patch, 
> OOZIE-2582-05.patch, OOZIE-2582-06.patch, OOZIE-2582-07.patch, 
> OOZIE-2582-08.patch, OOZIE-2582-09.patch
>
>
> Currently Oozie external child ids are populated into workflow bean, when the 
> job/action completes successfully. It should populate external child ids in 
> case of job failures as well.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OOZIE-2194) oozie job -kill doesn't work with spark action

[Abhishek Bafna (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-2194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Abhishek Bafna updated OOZIE-2194:
--
Attachment: OOZIE-2194-01.patch

Added 'spark.yarn.tags' as the '--conf' in the Spark argument list.

> oozie job -kill doesn't work with spark action
> --
>
> Key: OOZIE-2194
> URL: https://issues.apache.org/jira/browse/OOZIE-2194
> Project: Oozie
>  Issue Type: Bug
>Reporter: Thomas Graves
>Assignee: Abhishek Bafna
> Fix For: 4.3.0
>
> Attachments: OOZIE-2194-00.patch, OOZIE-2194-01.patch
>
>
> I was trying the spark action out with oozie and was able to launch fine. But 
> when I tried do use oozie job -kill it only killed the launcher job and not 
> the spark application itself.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2194) oozie job -kill doesn't work with spark action

[Abhishek Bafna (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15528442#comment-15528442
 ] 

Abhishek Bafna commented on OOZIE-2194:
---

Thanks [~rohini] and [~satishsaley].

> oozie job -kill doesn't work with spark action
> --
>
> Key: OOZIE-2194
> URL: https://issues.apache.org/jira/browse/OOZIE-2194
> Project: Oozie
>  Issue Type: Bug
>Reporter: Thomas Graves
>Assignee: Abhishek Bafna
> Fix For: 4.3.0
>
> Attachments: OOZIE-2194-00.patch, OOZIE-2194-01.patch
>
>
> I was trying the spark action out with oozie and was able to launch fine. But 
> when I tried do use oozie job -kill it only killed the launcher job and not 
> the spark application itself.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2573) dataset url which contains spaces can not be handled rightly

[Abhishek Bafna (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15528677#comment-15528677
 ] 

Abhishek Bafna commented on OOZIE-2573:
---

[~sjtufighter] can you please add a testcase also for the same? You can added 
the use case your are facing as the test case. Thanks.


> dataset url  which contains spaces can not be handled rightly
> -
>
> Key: OOZIE-2573
> URL: https://issues.apache.org/jira/browse/OOZIE-2573
> Project: Oozie
>  Issue Type: Bug
>  Components: core
>Affects Versions: 4.2.0
>Reporter: WangMeng
>Assignee: WangMeng
> Fix For: 4.3.0
>
> Attachments: OOZIE-2573.01.patch, OOZIE-2573.02.patch
>
>
> Currently , Oozie does not escape special character-spaces , this will lead 
> to dataset url  which contains spaces can not be handled rightly.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2613) Upgrade hive version from 0.13.1 to 1.2.0

[Abhishek Bafna (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15528880#comment-15528880
 ] 

Abhishek Bafna commented on OOZIE-2613:
---

Yes, i.e. the issue I faced and went for a workaround to get it working. Though 
I also did not like the solution and would favor a cleaner way than what I did. 

I am looking into if we can specify some property while starting the metastore 
server or property while working with database/tables/partitions.

> Upgrade hive version from 0.13.1 to 1.2.0
> -
>
> Key: OOZIE-2613
> URL: https://issues.apache.org/jira/browse/OOZIE-2613
> Project: Oozie
>  Issue Type: Bug
>  Components: core
>Reporter: Abhishek Bafna
>Assignee: Abhishek Bafna
>Priority: Blocker
> Fix For: 4.3.0
>
> Attachments: OOZIE-2613-00.patch, OOZIE-2613-01.patch, 
> OOZIE-2613-02.patch, OOZIE-2613-03.patch, OOZIE-2613-05.patch, 
> OOZIE-2613-06.patch, OOZIE-2613-07.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2225) Add wild card filter for gathering jobs

[Abhishek Bafna (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15529297#comment-15529297
 ] 

Abhishek Bafna commented on OOZIE-2225:
---

You should add the group field in the above RB. If you type 'oozie', it will 
auto complete it. Thanks.



> Add wild card filter for gathering jobs
> ---
>
> Key: OOZIE-2225
> URL: https://issues.apache.org/jira/browse/OOZIE-2225
> Project: Oozie
>  Issue Type: Improvement
>  Components: core
>Affects Versions: 4.1.0
>Reporter: Sai Chirravuri
>Assignee: Sai Chirravuri
> Attachments: OOZIE-2225-001.patch, OOZIE-2225-002.patch, 
> OOZIE-2225-003.patch, OOZIE-2225-004.patch, OOZIE-2225-005.patch
>
>
> Oozie currently supports name, user, group, status, frequency and unit as 
> filter params.
> Add wild card param that matches partially with name/user or complete match 
> with ID



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (OOZIE-2690) OOZIE NPE while executing kill()

[Abhishek Bafna (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-2690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Abhishek Bafna reassigned OOZIE-2690:
-

Assignee: Abhishek Bafna

> OOZIE NPE while executing kill()
> 
>
> Key: OOZIE-2690
> URL: https://issues.apache.org/jira/browse/OOZIE-2690
> Project: Oozie
>  Issue Type: Bug
>Reporter: Cheng Xu
>Assignee: Abhishek Bafna
>
> ION[027-160917110703953-oozie-oozi-W@java2] Exception while executing 
> kill(). Error Code [NullPointerException], Message[NullPointerException: null]
> org.apache.oozie.action.ActionExecutorException: NullPointerException: null
> at 
> org.apache.oozie.action.ActionExecutor.convertException(ActionExecutor.java:446)
> at 
> org.apache.oozie.action.hadoop.JavaActionExecutor.kill(JavaActionExecutor.java:1632)
> at 
> org.apache.oozie.command.wf.ActionKillXCommand.execute(ActionKillXCommand.java:146)
> at 
> org.apache.oozie.command.wf.ActionKillXCommand.execute(ActionKillXCommand.java:62)
> at org.apache.oozie.command.XCommand.call(XCommand.java:287)
> at 
> org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:331)
> at 
> org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:260)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at 
> org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:178)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.NullPointerException
> at org.apache.hadoop.mapreduce.TypeConverter.toYarn(TypeConverter.java:79)
> at 
> org.apache.hadoop.mapred.ClientServiceDelegate.(ClientServiceDelegate.java:120)
> at org.apache.hadoop.mapred.ClientCache.getClient(ClientCache.java:68)
> at org.apache.hadoop.mapred.YARNRunner.getJobStatus(YARNRunner.java:573)
> at org.apache.hadoop.mapreduce.Cluster.getJob(Cluster.java:184)
> at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:601)
> at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:599)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724)
> at 
> org.apache.hadoop.mapred.JobClient.getJobUsingCluster(JobClient.java:599)
> at org.apache.hadoop.mapred.JobClient.getJobInner(JobClient.java:609)
> at org.apache.hadoop.mapred.JobClient.getJob(JobClient.java:639)
> at 
> org.apache.oozie.action.hadoop.JavaActionExecutor.getJobClientWithRetries(JavaActionExecutor.java:1432)
> at 
> org.apache.oozie.action.hadoop.JavaActionExecutor.getRunningJob(JavaActionExecutor.java:1420)
> at 
> org.apache.oozie.action.hadoop.JavaActionExecutor.kill(JavaActionExecutor.java:1623)
> ... 10 more



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2613) Upgrade hive version from 0.13.1 to 1.2.0

[Abhishek Bafna (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15530481#comment-15530481
 ] 

Abhishek Bafna commented on OOZIE-2613:
---

Does these following documentation useful to this issue: 
https://cwiki.apache.org/confluence/display/Hive/HCatalog+Authorization
https://cwiki.apache.org/confluence/display/Hive/Permission+Inheritance+in+Hive

I tried setting {{hive.warehouse.subdir.inherit.perms}} in the 
{{initMetastoreServerConf}} and {{startMetastoreServer}} methods, but did not 
help.


> Upgrade hive version from 0.13.1 to 1.2.0
> -
>
> Key: OOZIE-2613
> URL: https://issues.apache.org/jira/browse/OOZIE-2613
> Project: Oozie
>  Issue Type: Bug
>  Components: core
>Reporter: Abhishek Bafna
>Assignee: Abhishek Bafna
>Priority: Blocker
> Fix For: 4.3.0
>
> Attachments: OOZIE-2613-00.patch, OOZIE-2613-01.patch, 
> OOZIE-2613-02.patch, OOZIE-2613-03.patch, OOZIE-2613-05.patch, 
> OOZIE-2613-06.patch, OOZIE-2613-07.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OOZIE-2613) Upgrade hive version from 0.13.1 to 1.2.0

[Abhishek Bafna (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-2613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Abhishek Bafna updated OOZIE-2613:
--
Attachment: OOZIE-2613-08.patch

Removed the workaround and submitting a cleaner patch. The 
{{TestFsActionExecutor#testDeleteHcat}} testcase issue is not fixed.

> Upgrade hive version from 0.13.1 to 1.2.0
> -
>
> Key: OOZIE-2613
> URL: https://issues.apache.org/jira/browse/OOZIE-2613
> Project: Oozie
>  Issue Type: Bug
>  Components: core
>Reporter: Abhishek Bafna
>Assignee: Abhishek Bafna
>Priority: Blocker
> Fix For: 4.3.0
>
> Attachments: OOZIE-2613-00.patch, OOZIE-2613-01.patch, 
> OOZIE-2613-02.patch, OOZIE-2613-03.patch, OOZIE-2613-05.patch, 
> OOZIE-2613-06.patch, OOZIE-2613-07.patch, OOZIE-2613-08.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (OOZIE-2673) Include XSD for shell-action:0.3 in documentation

[Abhishek Bafna (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-2673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Abhishek Bafna reassigned OOZIE-2673:
-

Assignee: Abhishek Bafna

> Include XSD for shell-action:0.3 in documentation
> -
>
> Key: OOZIE-2673
> URL: https://issues.apache.org/jira/browse/OOZIE-2673
> Project: Oozie
>  Issue Type: Bug
>Reporter: Satish Subhashrao Saley
>Assignee: Abhishek Bafna
>Priority: Minor
>  Labels: documentation, newbie
>
> We made following fields optional in shell-action:0.3. 
> {code}
> 
> 
> {code}
> Documentation need to be updated to include xsd for shell-action:0.3.
> [Here is the 
> code|https://github.com/apache/oozie/blob/master/docs/src/site/twiki/DG_ShellActionExtension.twiki]
>  for documentation page which needs an update.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OOZIE-2673) Include XSD for shell-action:0.3 in documentation

[Abhishek Bafna (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-2673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Abhishek Bafna updated OOZIE-2673:
--
Attachment: OOZIE-2673-00.patch

> Include XSD for shell-action:0.3 in documentation
> -
>
> Key: OOZIE-2673
> URL: https://issues.apache.org/jira/browse/OOZIE-2673
> Project: Oozie
>  Issue Type: Bug
>Reporter: Satish Subhashrao Saley
>Assignee: Abhishek Bafna
>Priority: Minor
>  Labels: documentation, newbie
> Attachments: OOZIE-2673-00.patch
>
>
> We made following fields optional in shell-action:0.3. 
> {code}
> 
> 
> {code}
> Documentation need to be updated to include xsd for shell-action:0.3.
> [Here is the 
> code|https://github.com/apache/oozie/blob/master/docs/src/site/twiki/DG_ShellActionExtension.twiki]
>  for documentation page which needs an update.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OOZIE-2673) Include XSD for shell-action:0.3 in documentation

[Abhishek Bafna (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-2673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Abhishek Bafna updated OOZIE-2673:
--
Fix Version/s: 4.3.0

> Include XSD for shell-action:0.3 in documentation
> -
>
> Key: OOZIE-2673
> URL: https://issues.apache.org/jira/browse/OOZIE-2673
> Project: Oozie
>  Issue Type: Bug
>Reporter: Satish Subhashrao Saley
>Assignee: Abhishek Bafna
>Priority: Minor
>  Labels: documentation, newbie
> Fix For: 4.3.0
>
> Attachments: OOZIE-2673-00.patch
>
>
> We made following fields optional in shell-action:0.3. 
> {code}
> 
> 
> {code}
> Documentation need to be updated to include xsd for shell-action:0.3.
> [Here is the 
> code|https://github.com/apache/oozie/blob/master/docs/src/site/twiki/DG_ShellActionExtension.twiki]
>  for documentation page which needs an update.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2690) OOZIE NPE while executing kill()

[Abhishek Bafna (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15532705#comment-15532705
 ] 

Abhishek Bafna commented on OOZIE-2690:
---

The scenario is, Fork has launched two actions e.g. {{action1}} and 
{{action2}}. For a reason {{action1}} fails to submits the launcher job to RM 
and {{action2}} get launched and fails in the launched job e.g. CNF exception. 
Now, as a action has failed, Fork will kill all the other actions launched as 
part of this Fork-Join. As the {{action1}} did not get submitted successfully 
so {{externalId}} will be null. i.e. when NPE happens in the 
{{JavaActionExecutor#getRunningJob}}.

> OOZIE NPE while executing kill()
> 
>
> Key: OOZIE-2690
> URL: https://issues.apache.org/jira/browse/OOZIE-2690
> Project: Oozie
>  Issue Type: Bug
>Reporter: Cheng Xu
>Assignee: Abhishek Bafna
>
> ION[027-160917110703953-oozie-oozi-W@java2] Exception while executing 
> kill(). Error Code [NullPointerException], Message[NullPointerException: null]
> org.apache.oozie.action.ActionExecutorException: NullPointerException: null
> at 
> org.apache.oozie.action.ActionExecutor.convertException(ActionExecutor.java:446)
> at 
> org.apache.oozie.action.hadoop.JavaActionExecutor.kill(JavaActionExecutor.java:1632)
> at 
> org.apache.oozie.command.wf.ActionKillXCommand.execute(ActionKillXCommand.java:146)
> at 
> org.apache.oozie.command.wf.ActionKillXCommand.execute(ActionKillXCommand.java:62)
> at org.apache.oozie.command.XCommand.call(XCommand.java:287)
> at 
> org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:331)
> at 
> org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:260)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at 
> org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:178)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.NullPointerException
> at org.apache.hadoop.mapreduce.TypeConverter.toYarn(TypeConverter.java:79)
> at 
> org.apache.hadoop.mapred.ClientServiceDelegate.(ClientServiceDelegate.java:120)
> at org.apache.hadoop.mapred.ClientCache.getClient(ClientCache.java:68)
> at org.apache.hadoop.mapred.YARNRunner.getJobStatus(YARNRunner.java:573)
> at org.apache.hadoop.mapreduce.Cluster.getJob(Cluster.java:184)
> at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:601)
> at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:599)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724)
> at 
> org.apache.hadoop.mapred.JobClient.getJobUsingCluster(JobClient.java:599)
> at org.apache.hadoop.mapred.JobClient.getJobInner(JobClient.java:609)
> at org.apache.hadoop.mapred.JobClient.getJob(JobClient.java:639)
> at 
> org.apache.oozie.action.hadoop.JavaActionExecutor.getJobClientWithRetries(JavaActionExecutor.java:1432)
> at 
> org.apache.oozie.action.hadoop.JavaActionExecutor.getRunningJob(JavaActionExecutor.java:1420)
> at 
> org.apache.oozie.action.hadoop.JavaActionExecutor.kill(JavaActionExecutor.java:1623)
> ... 10 more



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2613) Upgrade hive version from 0.13.1 to 1.2.0

[Abhishek Bafna (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15532868#comment-15532868
 ] 

Abhishek Bafna commented on OOZIE-2613:
---

There are three testcase which fails with the same Permission error:

Tests:
{noformat}
org.apache.oozie.action.hadoop.TestFsActionExecutor.testDeleteHcat

org.apache.oozie.command.coord.TestCoordRerunXCommand.testCoordRerunCleanupForHCat
org.apache.oozie.dependency.TestHCatURIHandler.testExists
{noformat}

Error:

{code}
org.apache.oozie.BaseEngineException: E0907: Failed to delete uri : E1501: 
Error in getting HCat Access [org.apache.hive.hcatalog.common.HCatException : 
9001 : Exception occurred while processing HCat request : MetaException while 
dropping partition.. Cause : MetaException(message:Got exception: 
org.apache.hadoop.security.AccessControlException Permission denied: user=test, 
access=WRITE, 
inode="/home/jenkins/jenkins-slave/workspace/oozie-trunk-precommit-build/core/target/test-data/oozietests/org.apache.oozie.command.coord.TestCoordRerunXCommand/testCoordRerunCleanupForHCat/mydb/mytable/ds=2009121411":jenkins:supergroup:drwxr-xr-x
  at 
{code}

> Upgrade hive version from 0.13.1 to 1.2.0
> -
>
> Key: OOZIE-2613
> URL: https://issues.apache.org/jira/browse/OOZIE-2613
> Project: Oozie
>  Issue Type: Bug
>  Components: core
>Reporter: Abhishek Bafna
>Assignee: Abhishek Bafna
>Priority: Blocker
> Fix For: 4.3.0
>
> Attachments: OOZIE-2613-00.patch, OOZIE-2613-01.patch, 
> OOZIE-2613-02.patch, OOZIE-2613-03.patch, OOZIE-2613-05.patch, 
> OOZIE-2613-06.patch, OOZIE-2613-07.patch, OOZIE-2613-08.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OOZIE-2690) OOZIE NPE while executing kill()

[Abhishek Bafna (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-2690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Abhishek Bafna updated OOZIE-2690:
--
Attachment: OOZIE-2690-00.patch

> OOZIE NPE while executing kill()
> 
>
> Key: OOZIE-2690
> URL: https://issues.apache.org/jira/browse/OOZIE-2690
> Project: Oozie
>  Issue Type: Bug
>Reporter: Cheng Xu
>Assignee: Abhishek Bafna
> Attachments: OOZIE-2690-00.patch
>
>
> ION[027-160917110703953-oozie-oozi-W@java2] Exception while executing 
> kill(). Error Code [NullPointerException], Message[NullPointerException: null]
> org.apache.oozie.action.ActionExecutorException: NullPointerException: null
> at 
> org.apache.oozie.action.ActionExecutor.convertException(ActionExecutor.java:446)
> at 
> org.apache.oozie.action.hadoop.JavaActionExecutor.kill(JavaActionExecutor.java:1632)
> at 
> org.apache.oozie.command.wf.ActionKillXCommand.execute(ActionKillXCommand.java:146)
> at 
> org.apache.oozie.command.wf.ActionKillXCommand.execute(ActionKillXCommand.java:62)
> at org.apache.oozie.command.XCommand.call(XCommand.java:287)
> at 
> org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:331)
> at 
> org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:260)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at 
> org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:178)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.NullPointerException
> at org.apache.hadoop.mapreduce.TypeConverter.toYarn(TypeConverter.java:79)
> at 
> org.apache.hadoop.mapred.ClientServiceDelegate.(ClientServiceDelegate.java:120)
> at org.apache.hadoop.mapred.ClientCache.getClient(ClientCache.java:68)
> at org.apache.hadoop.mapred.YARNRunner.getJobStatus(YARNRunner.java:573)
> at org.apache.hadoop.mapreduce.Cluster.getJob(Cluster.java:184)
> at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:601)
> at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:599)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724)
> at 
> org.apache.hadoop.mapred.JobClient.getJobUsingCluster(JobClient.java:599)
> at org.apache.hadoop.mapred.JobClient.getJobInner(JobClient.java:609)
> at org.apache.hadoop.mapred.JobClient.getJob(JobClient.java:639)
> at 
> org.apache.oozie.action.hadoop.JavaActionExecutor.getJobClientWithRetries(JavaActionExecutor.java:1432)
> at 
> org.apache.oozie.action.hadoop.JavaActionExecutor.getRunningJob(JavaActionExecutor.java:1420)
> at 
> org.apache.oozie.action.hadoop.JavaActionExecutor.kill(JavaActionExecutor.java:1623)
> ... 10 more



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2691) Show workflow action retry count to UI

[Abhishek Bafna (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15533674#comment-15533674
 ] 

Abhishek Bafna commented on OOZIE-2691:
---

Hi [~puru]
Like the {{userRetry}} workflow action also have {{retry}} count which 
represents or stores the retries for transient errors. The {{userRetry}} comes 
from, when user specifies max retry count and interval in the individual 
actions. So patch is intended to show the users; how many retries are done for 
the given action. I hope I got it correctly. If there is any use case you want 
to add, please provide. Thanks.

+1 {{non-binding}}

> Show workflow action retry count to UI
> --
>
> Key: OOZIE-2691
> URL: https://issues.apache.org/jira/browse/OOZIE-2691
> Project: Oozie
>  Issue Type: Bug
>Reporter: Purshotam Shah
>Assignee: Purshotam Shah
>Priority: Minor
> Attachments: OOZIE-2691-V1.patch, screenshot.png
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2613) Upgrade hive version from 0.13.1 to 1.2.0

[Abhishek Bafna (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15533688#comment-15533688
 ] 

Abhishek Bafna commented on OOZIE-2613:
---

[~gezapeti], [~asasvari], [~satishsaley], [~pbacsko], [~puru], 
[~jaydeepvishwakarma] Any ideas or help would be great to resolve the above 
testcase with Hive upgrade. Thanks.

> Upgrade hive version from 0.13.1 to 1.2.0
> -
>
> Key: OOZIE-2613
> URL: https://issues.apache.org/jira/browse/OOZIE-2613
> Project: Oozie
>  Issue Type: Bug
>  Components: core
>Reporter: Abhishek Bafna
>Assignee: Abhishek Bafna
>Priority: Blocker
> Fix For: 4.3.0
>
> Attachments: OOZIE-2613-00.patch, OOZIE-2613-01.patch, 
> OOZIE-2613-02.patch, OOZIE-2613-03.patch, OOZIE-2613-05.patch, 
> OOZIE-2613-06.patch, OOZIE-2613-07.patch, OOZIE-2613-08.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2693) SimpleHCatDependencyCache.removeMissingDependency can throw NPE

[Abhishek Bafna (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15535278#comment-15535278
 ] 

Abhishek Bafna commented on OOZIE-2693:
---

{{+1 non-binding}}

> SimpleHCatDependencyCache.removeMissingDependency can throw NPE
> ---
>
> Key: OOZIE-2693
> URL: https://issues.apache.org/jira/browse/OOZIE-2693
> Project: Oozie
>  Issue Type: Bug
>Reporter: Purshotam Shah
>Assignee: Purshotam Shah
> Attachments: OOZIE-2693-V1.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2694) Add logging for FsActionExecutor

[Abhishek Bafna (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15535297#comment-15535297
 ] 

Abhishek Bafna commented on OOZIE-2694:
---

{code}
LOG.debug("doOperations = [{0}], nameNodePath = [{1}], path = [{2}]", "mkdir", 
nameNodePath, path);
{code}

One trivial thing, {{doOperations}} can be {{doOperation}} as we are printing 
about one operation only. For all the log statements.

{{+1 non-binding}}

> Add logging for FsActionExecutor 
> -
>
> Key: OOZIE-2694
> URL: https://issues.apache.org/jira/browse/OOZIE-2694
> Project: Oozie
>  Issue Type: Bug
>Reporter: Purshotam Shah
>Assignee: Purshotam Shah
> Attachments: OOZIE-2694-V1.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2622) ExtJS 2.2 is no longer available

[Abhishek Bafna (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15536664#comment-15536664
 ] 

Abhishek Bafna commented on OOZIE-2622:
---

[~rohini] Can you please take a look at this? Thanks.

> ExtJS 2.2 is no longer available
> 
>
> Key: OOZIE-2622
> URL: https://issues.apache.org/jira/browse/OOZIE-2622
> Project: Oozie
>  Issue Type: Bug
>Affects Versions: 4.2.0
>Reporter: Sergey Svinarchuk
>Assignee: Robert Kanter
>Priority: Blocker
> Fix For: 4.3.0
>
> Attachments: OOZIE-2622.001.patch
>
>
> ExtJS 2.2 is no longer available by link from Oozie docs - 
> http://dev.sencha.com/deploy/ext-2.2.zip 
> (http://oozie.apache.org/docs/4.2.0/DG_QuickStart.html)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2690) OOZIE NPE while executing kill()

[Abhishek Bafna (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15536704#comment-15536704
 ] 

Abhishek Bafna commented on OOZIE-2690:
---

[~gezapeti] I wanted to write a testcase before submitting the patch, But do 
not know how to fail the job submission (JobClient#submitJob) for one action 
and not for the other action. We found this issue, during testing due to a 
issue into other component. The underlined issue was also intermittent in 
nature. That is the reason, I provided the scenario as comment for clarity of 
the issue and fix provided for it. Is there anything you will can be done, 
please do let me know. Thanks.

> OOZIE NPE while executing kill()
> 
>
> Key: OOZIE-2690
> URL: https://issues.apache.org/jira/browse/OOZIE-2690
> Project: Oozie
>  Issue Type: Bug
>Reporter: Cheng Xu
>Assignee: Abhishek Bafna
> Attachments: OOZIE-2690-00.patch
>
>
> ION[027-160917110703953-oozie-oozi-W@java2] Exception while executing 
> kill(). Error Code [NullPointerException], Message[NullPointerException: null]
> org.apache.oozie.action.ActionExecutorException: NullPointerException: null
> at 
> org.apache.oozie.action.ActionExecutor.convertException(ActionExecutor.java:446)
> at 
> org.apache.oozie.action.hadoop.JavaActionExecutor.kill(JavaActionExecutor.java:1632)
> at 
> org.apache.oozie.command.wf.ActionKillXCommand.execute(ActionKillXCommand.java:146)
> at 
> org.apache.oozie.command.wf.ActionKillXCommand.execute(ActionKillXCommand.java:62)
> at org.apache.oozie.command.XCommand.call(XCommand.java:287)
> at 
> org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:331)
> at 
> org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:260)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at 
> org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:178)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.NullPointerException
> at org.apache.hadoop.mapreduce.TypeConverter.toYarn(TypeConverter.java:79)
> at 
> org.apache.hadoop.mapred.ClientServiceDelegate.(ClientServiceDelegate.java:120)
> at org.apache.hadoop.mapred.ClientCache.getClient(ClientCache.java:68)
> at org.apache.hadoop.mapred.YARNRunner.getJobStatus(YARNRunner.java:573)
> at org.apache.hadoop.mapreduce.Cluster.getJob(Cluster.java:184)
> at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:601)
> at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:599)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724)
> at 
> org.apache.hadoop.mapred.JobClient.getJobUsingCluster(JobClient.java:599)
> at org.apache.hadoop.mapred.JobClient.getJobInner(JobClient.java:609)
> at org.apache.hadoop.mapred.JobClient.getJob(JobClient.java:639)
> at 
> org.apache.oozie.action.hadoop.JavaActionExecutor.getJobClientWithRetries(JavaActionExecutor.java:1432)
> at 
> org.apache.oozie.action.hadoop.JavaActionExecutor.getRunningJob(JavaActionExecutor.java:1420)
> at 
> org.apache.oozie.action.hadoop.JavaActionExecutor.kill(JavaActionExecutor.java:1623)
> ... 10 more



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2613) Upgrade hive version from 0.13.1 to 1.2.0

[Abhishek Bafna (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15536818#comment-15536818
 ] 

Abhishek Bafna commented on OOZIE-2613:
---

[~gezapeti] If we set this into the system property then it will be set for the 
rest of the testcase execution, which can cause the other testcases to fail. 
May be we can reset it to its original value at the end of the test. cc: 
[~rkanter]

> Upgrade hive version from 0.13.1 to 1.2.0
> -
>
> Key: OOZIE-2613
> URL: https://issues.apache.org/jira/browse/OOZIE-2613
> Project: Oozie
>  Issue Type: Bug
>  Components: core
>Reporter: Abhishek Bafna
>Assignee: Abhishek Bafna
>Priority: Blocker
> Fix For: 4.3.0
>
> Attachments: OOZIE-2613-00.patch, OOZIE-2613-01.patch, 
> OOZIE-2613-02.patch, OOZIE-2613-03.patch, OOZIE-2613-05.patch, 
> OOZIE-2613-06.patch, OOZIE-2613-07.patch, OOZIE-2613-08.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2613) Upgrade hive version from 0.13.1 to 1.2.0

[Abhishek Bafna (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15536857#comment-15536857
 ] 

Abhishek Bafna commented on OOZIE-2613:
---

[~rkanter] i.e. a great method, I guess. I will test it and then submit a 
patch, If it works. :)

> Upgrade hive version from 0.13.1 to 1.2.0
> -
>
> Key: OOZIE-2613
> URL: https://issues.apache.org/jira/browse/OOZIE-2613
> Project: Oozie
>  Issue Type: Bug
>  Components: core
>Reporter: Abhishek Bafna
>Assignee: Abhishek Bafna
>Priority: Blocker
> Fix For: 4.3.0
>
> Attachments: OOZIE-2613-00.patch, OOZIE-2613-01.patch, 
> OOZIE-2613-02.patch, OOZIE-2613-03.patch, OOZIE-2613-05.patch, 
> OOZIE-2613-06.patch, OOZIE-2613-07.patch, OOZIE-2613-08.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OOZIE-2613) Upgrade hive version from 0.13.1 to 1.2.0

[Abhishek Bafna (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-2613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Abhishek Bafna updated OOZIE-2613:
--
Attachment: OOZIE-2613-09.patch

> Upgrade hive version from 0.13.1 to 1.2.0
> -
>
> Key: OOZIE-2613
> URL: https://issues.apache.org/jira/browse/OOZIE-2613
> Project: Oozie
>  Issue Type: Bug
>  Components: core
>Reporter: Abhishek Bafna
>Assignee: Abhishek Bafna
>Priority: Blocker
> Fix For: 4.3.0
>
> Attachments: OOZIE-2613-00.patch, OOZIE-2613-01.patch, 
> OOZIE-2613-02.patch, OOZIE-2613-03.patch, OOZIE-2613-05.patch, 
> OOZIE-2613-06.patch, OOZIE-2613-07.patch, OOZIE-2613-08.patch, 
> OOZIE-2613-09.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2613) Upgrade hive version from 0.13.1 to 1.2.0

[Abhishek Bafna (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15536918#comment-15536918
 ] 

Abhishek Bafna commented on OOZIE-2613:
---

This fix works for other two testcases as well. The issue was that 
table/partition get create with the current system user and our testcase was 
using the 'test' user and they were failing due to permission issue. 

I agree for the point to do more investigation and fix them properly. If the 
lasted patch {{-09}} works, then we can decide whether we want to move ahead 
with it or not. Thanks for the feedback.

> Upgrade hive version from 0.13.1 to 1.2.0
> -
>
> Key: OOZIE-2613
> URL: https://issues.apache.org/jira/browse/OOZIE-2613
> Project: Oozie
>  Issue Type: Bug
>  Components: core
>Reporter: Abhishek Bafna
>Assignee: Abhishek Bafna
>Priority: Blocker
> Fix For: 4.3.0
>
> Attachments: OOZIE-2613-00.patch, OOZIE-2613-01.patch, 
> OOZIE-2613-02.patch, OOZIE-2613-03.patch, OOZIE-2613-05.patch, 
> OOZIE-2613-06.patch, OOZIE-2613-07.patch, OOZIE-2613-08.patch, 
> OOZIE-2613-09.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2696) Fix NPE when is provided

[Abhishek Bafna (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15542080#comment-15542080
 ] 

Abhishek Bafna commented on OOZIE-2696:
---

Is these two are related: OOZIE-2553.

> Fix NPE when  is provided
> --
>
> Key: OOZIE-2696
> URL: https://issues.apache.org/jira/browse/OOZIE-2696
> Project: Oozie
>  Issue Type: Sub-task
>Reporter: Peter Bacsko
>Assignee: Peter Bacsko
>
> On the OYA branch, we receive an NPE if there is a {{}} section 
> with a Java action (probably other actions are also affected, I've only seen 
> it with Java so far).
> The problem doesn't show up if we use the Hive action alone.
> {code}
> 2016-10-03 03:02:13,993 INFO org.apache.oozie.command.wf.ActionStartXCommand: 
> SERVER[oya-secure-1.gce.cloudera.com] USER[systest] GROUP[-] TOKEN[] 
> APP[Hive] JOB[000-161003023130362-oozie-oozi-W] 
> ACTION[000-161003023130362-oozie-oozi-W@java-fb27] Start action 
> [000-161003023130362-oozie-oozi-W@java-fb27] with user-retry state : 
> userRetryCount [0], userRetryMax [0], userRetryInterval [10]
> 2016-10-03 03:02:14,201 WARN org.apache.oozie.command.wf.ActionStartXCommand: 
> SERVER[oya-secure-1.gce.cloudera.com] USER[systest] GROUP[-] TOKEN[] 
> APP[Hive] JOB[000-161003023130362-oozie-oozi-W] 
> ACTION[000-161003023130362-oozie-oozi-W@java-fb27] Error starting action 
> [java-fb27]. ErrorType [ERROR], ErrorCode [JA021], Message [JA021: Could not 
> load credentials with name [null]].]
> org.apache.oozie.action.ActionExecutorException: JA021: Could not load 
> credentials with name [null]].
>   at 
> org.apache.oozie.action.hadoop.JavaActionExecutor.getCredProperties(JavaActionExecutor.java:1367)
>   at 
> org.apache.oozie.action.hadoop.JavaActionExecutor.getActionCredentialsProperties(JavaActionExecutor.java:1324)
>   at 
> org.apache.oozie.action.hadoop.JavaActionExecutor.setCredentialPropertyToActionConf(JavaActionExecutor.java:1260)
>   at 
> org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:1003)
>   at 
> org.apache.oozie.action.hadoop.JavaActionExecutor.start(JavaActionExecutor.java:1385)
>   at 
> org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:248)
>   at 
> org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:63)
>   at org.apache.oozie.command.XCommand.call(XCommand.java:287)
>   at 
> org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:331)
>   at 
> org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:260)
>   at java.util.concurrent.FutureTask.run(FutureTask.java:262)
>   at 
> org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:178)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>   at java.lang.Thread.run(Thread.java:745)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (OOZIE-2696) Fix NPE when is provided

[Abhishek Bafna (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15542080#comment-15542080
 ] 

Abhishek Bafna edited comment on OOZIE-2696 at 10/3/16 10:33 AM:
-

Are these two related: OOZIE-2553 and Current


was (Author: abhishekbafna):
Is these two are related: OOZIE-2553.

> Fix NPE when  is provided
> --
>
> Key: OOZIE-2696
> URL: https://issues.apache.org/jira/browse/OOZIE-2696
> Project: Oozie
>  Issue Type: Sub-task
>Reporter: Peter Bacsko
>Assignee: Peter Bacsko
>
> On the OYA branch, we receive an NPE if there is a {{}} section 
> with a Java action (probably other actions are also affected, I've only seen 
> it with Java so far).
> The problem doesn't show up if we use the Hive action alone.
> {code}
> 2016-10-03 03:02:13,993 INFO org.apache.oozie.command.wf.ActionStartXCommand: 
> SERVER[oya-secure-1.gce.cloudera.com] USER[systest] GROUP[-] TOKEN[] 
> APP[Hive] JOB[000-161003023130362-oozie-oozi-W] 
> ACTION[000-161003023130362-oozie-oozi-W@java-fb27] Start action 
> [000-161003023130362-oozie-oozi-W@java-fb27] with user-retry state : 
> userRetryCount [0], userRetryMax [0], userRetryInterval [10]
> 2016-10-03 03:02:14,201 WARN org.apache.oozie.command.wf.ActionStartXCommand: 
> SERVER[oya-secure-1.gce.cloudera.com] USER[systest] GROUP[-] TOKEN[] 
> APP[Hive] JOB[000-161003023130362-oozie-oozi-W] 
> ACTION[000-161003023130362-oozie-oozi-W@java-fb27] Error starting action 
> [java-fb27]. ErrorType [ERROR], ErrorCode [JA021], Message [JA021: Could not 
> load credentials with name [null]].]
> org.apache.oozie.action.ActionExecutorException: JA021: Could not load 
> credentials with name [null]].
>   at 
> org.apache.oozie.action.hadoop.JavaActionExecutor.getCredProperties(JavaActionExecutor.java:1367)
>   at 
> org.apache.oozie.action.hadoop.JavaActionExecutor.getActionCredentialsProperties(JavaActionExecutor.java:1324)
>   at 
> org.apache.oozie.action.hadoop.JavaActionExecutor.setCredentialPropertyToActionConf(JavaActionExecutor.java:1260)
>   at 
> org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:1003)
>   at 
> org.apache.oozie.action.hadoop.JavaActionExecutor.start(JavaActionExecutor.java:1385)
>   at 
> org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:248)
>   at 
> org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:63)
>   at org.apache.oozie.command.XCommand.call(XCommand.java:287)
>   at 
> org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:331)
>   at 
> org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:260)
>   at java.util.concurrent.FutureTask.run(FutureTask.java:262)
>   at 
> org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:178)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>   at java.lang.Thread.run(Thread.java:745)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2591) OYA: Java Action

[Abhishek Bafna (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15542347#comment-15542347
 ] 

Abhishek Bafna commented on OOZIE-2591:
---

[~gezapeti] Is the patch committed to {{oya}}? As the issue us resolved but I 
do not see the changes in branch. Same for bunch of other OYA related issue. 
Thanks.

> OYA: Java Action
> 
>
> Key: OOZIE-2591
> URL: https://issues.apache.org/jira/browse/OOZIE-2591
> Project: Oozie
>  Issue Type: Sub-task
>Affects Versions: oya
>Reporter: Robert Kanter
>Assignee: Peter Cseh
> Fix For: oya
>
> Attachments: OOZIE-2591.01.patch, OOZIE-2591.02.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2591) OYA: Java Action

[Abhishek Bafna (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15542713#comment-15542713
 ] 

Abhishek Bafna commented on OOZIE-2591:
---

Thanks for the reply and information. That is really great work and news. 
Congratulations to both of you. [[~gezapeti] & [~pbacsko]] :)

> OYA: Java Action
> 
>
> Key: OOZIE-2591
> URL: https://issues.apache.org/jira/browse/OOZIE-2591
> Project: Oozie
>  Issue Type: Sub-task
>Affects Versions: oya
>Reporter: Robert Kanter
>Assignee: Peter Cseh
> Fix For: oya
>
> Attachments: OOZIE-2591.01.patch, OOZIE-2591.02.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (OOZIE-2701) Oozie to support Multiple HCatalog URIs

[Abhishek Bafna (JIRA)]

Abhishek Bafna created OOZIE-2701:
-

 Summary: Oozie to support Multiple HCatalog URIs
 Key: OOZIE-2701
 URL: https://issues.apache.org/jira/browse/OOZIE-2701
 Project: Oozie
  Issue Type: New Feature
  Components: core
Reporter: Abhishek Bafna
Assignee: Abhishek Bafna






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (OOZIE-2393) Allow table drop in hcat prepare

[Abhishek Bafna (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-2393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Abhishek Bafna reassigned OOZIE-2393:
-

Assignee: Abhishek Bafna

> Allow table drop in hcat prepare
> 
>
> Key: OOZIE-2393
> URL: https://issues.apache.org/jira/browse/OOZIE-2393
> Project: Oozie
>  Issue Type: Improvement
>  Components: core
>Affects Versions: 4.1.0, 4.2.0
>Reporter: Muhammad Ehsan ul Haque
>Assignee: Abhishek Bafna
>Priority: Minor
>  Labels: hcatalog, prepare
> Fix For: 5.0.0
>
>
> The hcat prepare only allows to drop partitions. It would be nice to also 
> allow dropping of table, depending on the URL. The current format of the URL 
> is
> {code}hcat://[metastore server]:[port]/[database name]/[table 
> name]/[partkey1]=[value];[partkey2]=[value]{code}
> where at least one partition must be provided, otherwise the prepare step 
> fails with the fololwing exception.
> {code}
> Starting the execution of prepare actions
> Creating HCatClient for user=ehsan.haq (auth:SIMPLE) and 
> server=thrift://datavault-prod-app2.internal.machines:9083
> Prepare execution in the Launcher Mapper has failed
> Failing Oozie Launcher, Main class 
> [org.apache.oozie.action.hadoop.SqoopMain], exception invoking main(), Error 
> trying to drop 
> hcat://datavault-prod-app2.internal.machines:9083/test_rdbms_import_2015110600/test
> org.apache.oozie.action.hadoop.LauncherException: Error trying to drop 
> hcat://datavault-prod-app2.internal.machines:9083/test_rdbms_import_2015110600/test
> at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:178)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:450)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
> Caused by: org.apache.oozie.action.hadoop.LauncherException: Error trying to 
> drop 
> hcat://datavault-prod-app2.internal.machines:9083/test_rdbms_import_2015110600/test
> at 
> org.apache.oozie.action.hadoop.HCatLauncherURIHandler.delete(HCatLauncherURIHandler.java:64)
> at 
> org.apache.oozie.action.hadoop.PrepareActionsDriver.execute(PrepareActionsDriver.java:89)
> at 
> org.apache.oozie.action.hadoop.PrepareActionsDriver.doOperations(PrepareActionsDriver.java:67)
> at 
> org.apache.oozie.action.hadoop.LauncherMapper.executePrepare(LauncherMapper.java:446)
> at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:174)
> ... 8 more
> Caused by: java.net.URISyntaxException: URI path is not in expected format: 
> hcat://datavault-prod-app2.internal.machines:9083/test_rdbms_import_2015110600/test
> at org.apache.oozie.util.HCatURI.parse(HCatURI.java:66)
> at org.apache.oozie.util.HCatURI.(HCatURI.java:52)
> at org.apache.oozie.util.HCatURI.(HCatURI.java:48)
> at 
> org.apache.oozie.action.hadoop.HCatLauncherURIHandler.delete(HCatLauncherURIHandler.java:52)
> ... 12 more
> Oozie Launcher failed, finishing Hadoop job gracefully
> {code}
> h3. Suggestion
> In the URL if the partition is not provided then it should delete the entire 
> table.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OOZIE-2393) Allow table drop in hcat prepare

[Abhishek Bafna (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-2393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Abhishek Bafna updated OOZIE-2393:
--
Attachment: OOZIE-2393-00.patch

> Allow table drop in hcat prepare
> 
>
> Key: OOZIE-2393
> URL: https://issues.apache.org/jira/browse/OOZIE-2393
> Project: Oozie
>  Issue Type: Improvement
>  Components: core
>Affects Versions: 4.1.0, 4.2.0
>Reporter: Muhammad Ehsan ul Haque
>Assignee: Abhishek Bafna
>Priority: Minor
>  Labels: hcatalog, prepare
> Fix For: 5.0.0
>
> Attachments: OOZIE-2393-00.patch
>
>
> The hcat prepare only allows to drop partitions. It would be nice to also 
> allow dropping of table, depending on the URL. The current format of the URL 
> is
> {code}hcat://[metastore server]:[port]/[database name]/[table 
> name]/[partkey1]=[value];[partkey2]=[value]{code}
> where at least one partition must be provided, otherwise the prepare step 
> fails with the fololwing exception.
> {code}
> Starting the execution of prepare actions
> Creating HCatClient for user=ehsan.haq (auth:SIMPLE) and 
> server=thrift://datavault-prod-app2.internal.machines:9083
> Prepare execution in the Launcher Mapper has failed
> Failing Oozie Launcher, Main class 
> [org.apache.oozie.action.hadoop.SqoopMain], exception invoking main(), Error 
> trying to drop 
> hcat://datavault-prod-app2.internal.machines:9083/test_rdbms_import_2015110600/test
> org.apache.oozie.action.hadoop.LauncherException: Error trying to drop 
> hcat://datavault-prod-app2.internal.machines:9083/test_rdbms_import_2015110600/test
> at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:178)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:450)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
> Caused by: org.apache.oozie.action.hadoop.LauncherException: Error trying to 
> drop 
> hcat://datavault-prod-app2.internal.machines:9083/test_rdbms_import_2015110600/test
> at 
> org.apache.oozie.action.hadoop.HCatLauncherURIHandler.delete(HCatLauncherURIHandler.java:64)
> at 
> org.apache.oozie.action.hadoop.PrepareActionsDriver.execute(PrepareActionsDriver.java:89)
> at 
> org.apache.oozie.action.hadoop.PrepareActionsDriver.doOperations(PrepareActionsDriver.java:67)
> at 
> org.apache.oozie.action.hadoop.LauncherMapper.executePrepare(LauncherMapper.java:446)
> at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:174)
> ... 8 more
> Caused by: java.net.URISyntaxException: URI path is not in expected format: 
> hcat://datavault-prod-app2.internal.machines:9083/test_rdbms_import_2015110600/test
> at org.apache.oozie.util.HCatURI.parse(HCatURI.java:66)
> at org.apache.oozie.util.HCatURI.(HCatURI.java:52)
> at org.apache.oozie.util.HCatURI.(HCatURI.java:48)
> at 
> org.apache.oozie.action.hadoop.HCatLauncherURIHandler.delete(HCatLauncherURIHandler.java:52)
> ... 12 more
> Oozie Launcher failed, finishing Hadoop job gracefully
> {code}
> h3. Suggestion
> In the URL if the partition is not provided then it should delete the entire 
> table.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OOZIE-2393) Allow table drop in hcat prepare

[Abhishek Bafna (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-2393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Abhishek Bafna updated OOZIE-2393:
--
Attachment: (was: OOZIE-2393-00.patch)

> Allow table drop in hcat prepare
> 
>
> Key: OOZIE-2393
> URL: https://issues.apache.org/jira/browse/OOZIE-2393
> Project: Oozie
>  Issue Type: Improvement
>  Components: core
>Affects Versions: 4.1.0, 4.2.0
>Reporter: Muhammad Ehsan ul Haque
>Assignee: Abhishek Bafna
>Priority: Minor
>  Labels: hcatalog, prepare
> Fix For: 5.0.0
>
>
> The hcat prepare only allows to drop partitions. It would be nice to also 
> allow dropping of table, depending on the URL. The current format of the URL 
> is
> {code}hcat://[metastore server]:[port]/[database name]/[table 
> name]/[partkey1]=[value];[partkey2]=[value]{code}
> where at least one partition must be provided, otherwise the prepare step 
> fails with the fololwing exception.
> {code}
> Starting the execution of prepare actions
> Creating HCatClient for user=ehsan.haq (auth:SIMPLE) and 
> server=thrift://datavault-prod-app2.internal.machines:9083
> Prepare execution in the Launcher Mapper has failed
> Failing Oozie Launcher, Main class 
> [org.apache.oozie.action.hadoop.SqoopMain], exception invoking main(), Error 
> trying to drop 
> hcat://datavault-prod-app2.internal.machines:9083/test_rdbms_import_2015110600/test
> org.apache.oozie.action.hadoop.LauncherException: Error trying to drop 
> hcat://datavault-prod-app2.internal.machines:9083/test_rdbms_import_2015110600/test
> at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:178)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:450)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
> Caused by: org.apache.oozie.action.hadoop.LauncherException: Error trying to 
> drop 
> hcat://datavault-prod-app2.internal.machines:9083/test_rdbms_import_2015110600/test
> at 
> org.apache.oozie.action.hadoop.HCatLauncherURIHandler.delete(HCatLauncherURIHandler.java:64)
> at 
> org.apache.oozie.action.hadoop.PrepareActionsDriver.execute(PrepareActionsDriver.java:89)
> at 
> org.apache.oozie.action.hadoop.PrepareActionsDriver.doOperations(PrepareActionsDriver.java:67)
> at 
> org.apache.oozie.action.hadoop.LauncherMapper.executePrepare(LauncherMapper.java:446)
> at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:174)
> ... 8 more
> Caused by: java.net.URISyntaxException: URI path is not in expected format: 
> hcat://datavault-prod-app2.internal.machines:9083/test_rdbms_import_2015110600/test
> at org.apache.oozie.util.HCatURI.parse(HCatURI.java:66)
> at org.apache.oozie.util.HCatURI.(HCatURI.java:52)
> at org.apache.oozie.util.HCatURI.(HCatURI.java:48)
> at 
> org.apache.oozie.action.hadoop.HCatLauncherURIHandler.delete(HCatLauncherURIHandler.java:52)
> ... 12 more
> Oozie Launcher failed, finishing Hadoop job gracefully
> {code}
> h3. Suggestion
> In the URL if the partition is not provided then it should delete the entire 
> table.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OOZIE-2393) Allow table drop in hcat prepare

[Abhishek Bafna (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-2393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Abhishek Bafna updated OOZIE-2393:
--
Attachment: OOZIE-2393-00.patch

> Allow table drop in hcat prepare
> 
>
> Key: OOZIE-2393
> URL: https://issues.apache.org/jira/browse/OOZIE-2393
> Project: Oozie
>  Issue Type: Improvement
>  Components: core
>Affects Versions: 4.1.0, 4.2.0
>Reporter: Muhammad Ehsan ul Haque
>Assignee: Abhishek Bafna
>Priority: Minor
>  Labels: hcatalog, prepare
> Fix For: 5.0.0
>
> Attachments: OOZIE-2393-00.patch
>
>
> The hcat prepare only allows to drop partitions. It would be nice to also 
> allow dropping of table, depending on the URL. The current format of the URL 
> is
> {code}hcat://[metastore server]:[port]/[database name]/[table 
> name]/[partkey1]=[value];[partkey2]=[value]{code}
> where at least one partition must be provided, otherwise the prepare step 
> fails with the fololwing exception.
> {code}
> Starting the execution of prepare actions
> Creating HCatClient for user=ehsan.haq (auth:SIMPLE) and 
> server=thrift://datavault-prod-app2.internal.machines:9083
> Prepare execution in the Launcher Mapper has failed
> Failing Oozie Launcher, Main class 
> [org.apache.oozie.action.hadoop.SqoopMain], exception invoking main(), Error 
> trying to drop 
> hcat://datavault-prod-app2.internal.machines:9083/test_rdbms_import_2015110600/test
> org.apache.oozie.action.hadoop.LauncherException: Error trying to drop 
> hcat://datavault-prod-app2.internal.machines:9083/test_rdbms_import_2015110600/test
> at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:178)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:450)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
> Caused by: org.apache.oozie.action.hadoop.LauncherException: Error trying to 
> drop 
> hcat://datavault-prod-app2.internal.machines:9083/test_rdbms_import_2015110600/test
> at 
> org.apache.oozie.action.hadoop.HCatLauncherURIHandler.delete(HCatLauncherURIHandler.java:64)
> at 
> org.apache.oozie.action.hadoop.PrepareActionsDriver.execute(PrepareActionsDriver.java:89)
> at 
> org.apache.oozie.action.hadoop.PrepareActionsDriver.doOperations(PrepareActionsDriver.java:67)
> at 
> org.apache.oozie.action.hadoop.LauncherMapper.executePrepare(LauncherMapper.java:446)
> at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:174)
> ... 8 more
> Caused by: java.net.URISyntaxException: URI path is not in expected format: 
> hcat://datavault-prod-app2.internal.machines:9083/test_rdbms_import_2015110600/test
> at org.apache.oozie.util.HCatURI.parse(HCatURI.java:66)
> at org.apache.oozie.util.HCatURI.(HCatURI.java:52)
> at org.apache.oozie.util.HCatURI.(HCatURI.java:48)
> at 
> org.apache.oozie.action.hadoop.HCatLauncherURIHandler.delete(HCatLauncherURIHandler.java:52)
> ... 12 more
> Oozie Launcher failed, finishing Hadoop job gracefully
> {code}
> h3. Suggestion
> In the URL if the partition is not provided then it should delete the entire 
> table.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2613) Upgrade hive version from 0.13.1 to 1.2.0

[Abhishek Bafna (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15551869#comment-15551869
 ] 

Abhishek Bafna commented on OOZIE-2613:
---

The directory structure for the partitions are created in the 
{{MiniHCatServer#createPartitionDir}} method. I replaced the code with below 
but it is still giving the same error.

{code}
public String createPartitionDir(String db, String table, String 
partitionSpec, String dbLocation) throws Exception {
FileSystem.get(hadoopConf).mkdirs(new Path(dbLocation), 
FsPermission.valueOf("-rwxrwxrwx"));
FileSystem.get(hadoopConf).setOwner(new Path(dbLocation), "test", 
"test");
String dir = getPartitionDir(db, table, partitionSpec, dbLocation);
FileSystem.get(hadoopConf).mkdirs(new Path(dir), 
FsPermission.valueOf("-rwxrwxrwx"));
FileSystem.get(hadoopConf).setOwner(new Path(dir), "test", "test");
return dir;
}
{code}

> Upgrade hive version from 0.13.1 to 1.2.0
> -
>
> Key: OOZIE-2613
> URL: https://issues.apache.org/jira/browse/OOZIE-2613
> Project: Oozie
>  Issue Type: Bug
>  Components: core
>Reporter: Abhishek Bafna
>Assignee: Abhishek Bafna
>Priority: Blocker
> Fix For: 4.3.0
>
> Attachments: OOZIE-2613-00.patch, OOZIE-2613-01.patch, 
> OOZIE-2613-02.patch, OOZIE-2613-03.patch, OOZIE-2613-05.patch, 
> OOZIE-2613-06.patch, OOZIE-2613-07.patch, OOZIE-2613-08.patch, 
> OOZIE-2613-09.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OOZIE-2393) Allow table drop in hcat prepare

[Abhishek Bafna (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-2393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Abhishek Bafna updated OOZIE-2393:
--
Attachment: OOZIE-2393-01.patch

> Allow table drop in hcat prepare
> 
>
> Key: OOZIE-2393
> URL: https://issues.apache.org/jira/browse/OOZIE-2393
> Project: Oozie
>  Issue Type: Improvement
>  Components: core
>Affects Versions: 4.1.0, 4.2.0
>Reporter: Muhammad Ehsan ul Haque
>Assignee: Abhishek Bafna
>Priority: Minor
>  Labels: hcatalog, prepare
> Fix For: 5.0.0
>
> Attachments: OOZIE-2393-00.patch, OOZIE-2393-01.patch
>
>
> The hcat prepare only allows to drop partitions. It would be nice to also 
> allow dropping of table, depending on the URL. The current format of the URL 
> is
> {code}hcat://[metastore server]:[port]/[database name]/[table 
> name]/[partkey1]=[value];[partkey2]=[value]{code}
> where at least one partition must be provided, otherwise the prepare step 
> fails with the fololwing exception.
> {code}
> Starting the execution of prepare actions
> Creating HCatClient for user=ehsan.haq (auth:SIMPLE) and 
> server=thrift://datavault-prod-app2.internal.machines:9083
> Prepare execution in the Launcher Mapper has failed
> Failing Oozie Launcher, Main class 
> [org.apache.oozie.action.hadoop.SqoopMain], exception invoking main(), Error 
> trying to drop 
> hcat://datavault-prod-app2.internal.machines:9083/test_rdbms_import_2015110600/test
> org.apache.oozie.action.hadoop.LauncherException: Error trying to drop 
> hcat://datavault-prod-app2.internal.machines:9083/test_rdbms_import_2015110600/test
> at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:178)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:450)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
> Caused by: org.apache.oozie.action.hadoop.LauncherException: Error trying to 
> drop 
> hcat://datavault-prod-app2.internal.machines:9083/test_rdbms_import_2015110600/test
> at 
> org.apache.oozie.action.hadoop.HCatLauncherURIHandler.delete(HCatLauncherURIHandler.java:64)
> at 
> org.apache.oozie.action.hadoop.PrepareActionsDriver.execute(PrepareActionsDriver.java:89)
> at 
> org.apache.oozie.action.hadoop.PrepareActionsDriver.doOperations(PrepareActionsDriver.java:67)
> at 
> org.apache.oozie.action.hadoop.LauncherMapper.executePrepare(LauncherMapper.java:446)
> at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:174)
> ... 8 more
> Caused by: java.net.URISyntaxException: URI path is not in expected format: 
> hcat://datavault-prod-app2.internal.machines:9083/test_rdbms_import_2015110600/test
> at org.apache.oozie.util.HCatURI.parse(HCatURI.java:66)
> at org.apache.oozie.util.HCatURI.(HCatURI.java:52)
> at org.apache.oozie.util.HCatURI.(HCatURI.java:48)
> at 
> org.apache.oozie.action.hadoop.HCatLauncherURIHandler.delete(HCatLauncherURIHandler.java:52)
> ... 12 more
> Oozie Launcher failed, finishing Hadoop job gracefully
> {code}
> h3. Suggestion
> In the URL if the partition is not provided then it should delete the entire 
> table.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2393) Allow table drop in hcat prepare

[Abhishek Bafna (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15554949#comment-15554949
 ] 

Abhishek Bafna commented on OOZIE-2393:
---

Thank you [~gezapeti] for the feedback and review.

> Allow table drop in hcat prepare
> 
>
> Key: OOZIE-2393
> URL: https://issues.apache.org/jira/browse/OOZIE-2393
> Project: Oozie
>  Issue Type: Improvement
>  Components: core
>Affects Versions: 4.1.0, 4.2.0
>Reporter: Muhammad Ehsan ul Haque
>Assignee: Abhishek Bafna
>Priority: Minor
>  Labels: hcatalog, prepare
> Fix For: 5.0.0
>
> Attachments: OOZIE-2393-00.patch, OOZIE-2393-01.patch
>
>
> The hcat prepare only allows to drop partitions. It would be nice to also 
> allow dropping of table, depending on the URL. The current format of the URL 
> is
> {code}hcat://[metastore server]:[port]/[database name]/[table 
> name]/[partkey1]=[value];[partkey2]=[value]{code}
> where at least one partition must be provided, otherwise the prepare step 
> fails with the fololwing exception.
> {code}
> Starting the execution of prepare actions
> Creating HCatClient for user=ehsan.haq (auth:SIMPLE) and 
> server=thrift://datavault-prod-app2.internal.machines:9083
> Prepare execution in the Launcher Mapper has failed
> Failing Oozie Launcher, Main class 
> [org.apache.oozie.action.hadoop.SqoopMain], exception invoking main(), Error 
> trying to drop 
> hcat://datavault-prod-app2.internal.machines:9083/test_rdbms_import_2015110600/test
> org.apache.oozie.action.hadoop.LauncherException: Error trying to drop 
> hcat://datavault-prod-app2.internal.machines:9083/test_rdbms_import_2015110600/test
> at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:178)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:450)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
> Caused by: org.apache.oozie.action.hadoop.LauncherException: Error trying to 
> drop 
> hcat://datavault-prod-app2.internal.machines:9083/test_rdbms_import_2015110600/test
> at 
> org.apache.oozie.action.hadoop.HCatLauncherURIHandler.delete(HCatLauncherURIHandler.java:64)
> at 
> org.apache.oozie.action.hadoop.PrepareActionsDriver.execute(PrepareActionsDriver.java:89)
> at 
> org.apache.oozie.action.hadoop.PrepareActionsDriver.doOperations(PrepareActionsDriver.java:67)
> at 
> org.apache.oozie.action.hadoop.LauncherMapper.executePrepare(LauncherMapper.java:446)
> at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:174)
> ... 8 more
> Caused by: java.net.URISyntaxException: URI path is not in expected format: 
> hcat://datavault-prod-app2.internal.machines:9083/test_rdbms_import_2015110600/test
> at org.apache.oozie.util.HCatURI.parse(HCatURI.java:66)
> at org.apache.oozie.util.HCatURI.(HCatURI.java:52)
> at org.apache.oozie.util.HCatURI.(HCatURI.java:48)
> at 
> org.apache.oozie.action.hadoop.HCatLauncherURIHandler.delete(HCatLauncherURIHandler.java:52)
> ... 12 more
> Oozie Launcher failed, finishing Hadoop job gracefully
> {code}
> h3. Suggestion
> In the URL if the partition is not provided then it should delete the entire 
> table.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2613) Upgrade hive version from 0.13.1 to 1.2.0

[Abhishek Bafna (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15556946#comment-15556946
 ] 

Abhishek Bafna commented on OOZIE-2613:
---

[~rkanter] For three testcase patch uses {{XTestCase#setSystemProperty}} to set 
the current user, because with the {{test}} user they are failing with 
{{Permission denied}}. Otherwise it is ready. Thanks.

> Upgrade hive version from 0.13.1 to 1.2.0
> -
>
> Key: OOZIE-2613
> URL: https://issues.apache.org/jira/browse/OOZIE-2613
> Project: Oozie
>  Issue Type: Bug
>  Components: core
>Reporter: Abhishek Bafna
>Assignee: Abhishek Bafna
>Priority: Blocker
> Fix For: 4.3.0
>
> Attachments: OOZIE-2613-00.patch, OOZIE-2613-01.patch, 
> OOZIE-2613-02.patch, OOZIE-2613-03.patch, OOZIE-2613-05.patch, 
> OOZIE-2613-06.patch, OOZIE-2613-07.patch, OOZIE-2613-08.patch, 
> OOZIE-2613-09.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2616) Add Tez profile for Hive and Pig sharelibs

[Abhishek Bafna (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15558379#comment-15558379
 ] 

Abhishek Bafna commented on OOZIE-2616:
---

{{+1 non-binding}}.
[~poeppt] The patch requires rebase. Thanks.


> Add Tez profile for Hive and Pig sharelibs
> --
>
> Key: OOZIE-2616
> URL: https://issues.apache.org/jira/browse/OOZIE-2616
> Project: Oozie
>  Issue Type: Improvement
>Affects Versions: trunk
>Reporter: Thomas Poepping
>Assignee: Thomas Poepping
> Attachments: OOZIE-2616-1.patch, OOZIE-2616-2.patch
>
>
> More recent versions of Hive and Pig depend on Tez as their default execution 
> engine. As the sharelib for those applications stands today, those 
> applications are unable to run Tez jobs. This patch adds an optional profile 
> to the root pom that will include the proper Tez dependencies into those 
> sharelibs, while excluding problematic jars to keep the sharelibs from having 
> classpath duplication issues. *Note*: this patch will only be required for 
> those clients who specify their own, newer versions of Hive or Pig. This 
> change is not required for the current supported versions of those 
> applications (hence why the profile is off by default).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OOZIE-2629) Get OozieShareLibCLI to perform a final rename to destPath when creating a Sharelib

[Abhishek Bafna (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-2629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Abhishek Bafna updated OOZIE-2629:
--
Fix Version/s: (was: 4.3.0)
   5.0.0

> Get OozieShareLibCLI to perform a final rename to destPath when creating a 
> Sharelib
> ---
>
> Key: OOZIE-2629
> URL: https://issues.apache.org/jira/browse/OOZIE-2629
> Project: Oozie
>  Issue Type: Improvement
>Reporter: Yishan Yang
>Priority: Minor
> Fix For: 5.0.0
>
> Attachments: OOZIE-2629.PATCH
>
>
> Change the OozieShareLibCLI to perform a final 'rename' to avoid a condition 
> where 'half' of the sharelib could exist during sharelib installation, if 
> Oozie attempts to run a workflow at the same time as the sharelib 
> installation. Oozie will only start if the sharelib exists, but this by 
> itself is not enough since the sharelib could be in an unusable state.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OOZIE-2518) Oozie is creating many files and directories in /tmp per day

[Abhishek Bafna (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-2518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Abhishek Bafna updated OOZIE-2518:
--
Fix Version/s: (was: 4.3.0)
   5.0.0

> Oozie is creating many files and directories in /tmp per day
> 
>
> Key: OOZIE-2518
> URL: https://issues.apache.org/jira/browse/OOZIE-2518
> Project: Oozie
>  Issue Type: Improvement
>Affects Versions: 4.2.0
>Reporter: Sergey Svinarchuk
>Assignee: Sergey Svinarchuk
>  Labels: patch
> Fix For: 5.0.0
>
> Attachments: OOZIE-2518.patch
>
>
> Oozie generate many tmp files, I have over 2000 files in /tmp from oozie, 
> that have been created over the last 7 days: 
> {code}
> drwxr-xr-x 3 root root   22 Apr 22 14:19 /tmp/oozie-war-packing-24575
> drwxr-xr-x 3 root root   22 Apr 22 14:15 /tmp/oozie-war-packing-12211
> drwxr-xr-x 3 root root   22 Apr 22 14:10 /tmp/oozie-war-packing-32238
> drwxr-xr-x 2 test test6 Apr 22 13:54 
> /tmp/oozie-test8457897583638353023.dir
> -rw-r--r-- 1 test test0 Apr 22 13:54 /tmp/ooziedb-2680155694357966339.sql
> drwxr-xr-x 2 test test6 Apr 22 13:53 /tmp/oozie-test142692242798969000.dir
> -rw-r--r-- 1 test test0 Apr 22 13:53 /tmp/ooziedb-9104680045020610294.sql
> drwxr-xr-x 2 test test6 Apr 22 13:53 
> /tmp/oozie-test5885601734493698256.dir
> -rw-r--r-- 1 test test0 Apr 22 13:53 /tmp/ooziedb-4675739133078261538.sql
> drwxr-xr-x 2 test test6 Apr 22 13:53 
> /tmp/oozie-test5565479595579419225.dir
> [root@test-node ext-conf]# ls -ldt /tmp/ooz* | tail
> drwxr-xr-x 3 root root   22 Apr 13 08:17 /tmp/oozie-war-packing-22705
> drwxr-xr-x 3 root root   22 Apr 13 08:16 /tmp/oozie-war-packing-18292
> drwxr-xr-x 3 root root   22 Apr 13 08:09 /tmp/oozie-war-packing-20289
> drwxr-xr-x 3 root root   22 Apr 13 08:09 /tmp/oozie-war-packing-18417
> drwxr-xr-x 3 root root   22 Apr 13 08:08 /tmp/oozie-war-packing-14712
> drwxr-xr-x 3 root root   22 Apr 13 08:05 /tmp/oozie-war-packing-25206
> drwxr-xr-x 3 root root   22 Apr 13 07:53 /tmp/oozie-war-packing-968
> drwxr-xr-x 3 root root   22 Apr 13 07:52 /tmp/oozie-war-packing-27771
> drwxr-xr-x 3 root root   22 Apr 13 07:45 /tmp/oozie-war-packing-30342
> drwxr-xr-x 3 root root   22 Apr 13 07:45 /tmp/oozie-war-packing-29086
> {code}
> This files can be moved into oozie tmp dir (something like this 
> /tmp/oozieTmp) and also we could cleanup old files



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OOZIE-2573) dataset url which contains spaces can not be handled rightly

[Abhishek Bafna (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-2573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Abhishek Bafna updated OOZIE-2573:
--
Fix Version/s: (was: 4.3.0)
   5.0.0

> dataset url  which contains spaces can not be handled rightly
> -
>
> Key: OOZIE-2573
> URL: https://issues.apache.org/jira/browse/OOZIE-2573
> Project: Oozie
>  Issue Type: Bug
>  Components: core
>Affects Versions: 4.2.0
>Reporter: WangMeng
>Assignee: WangMeng
> Fix For: 5.0.0
>
> Attachments: OOZIE-2573.01.patch, OOZIE-2573.02.patch
>
>
> Currently , Oozie does not escape special character-spaces , this will lead 
> to dataset url  which contains spaces can not be handled rightly.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OOZIE-2231) Upgrade curator to latest version 2.11.0

[Abhishek Bafna (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-2231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Abhishek Bafna updated OOZIE-2231:
--
Fix Version/s: (was: 4.3.0)
   5.0.0

> Upgrade curator to latest version 2.11.0
> 
>
> Key: OOZIE-2231
> URL: https://issues.apache.org/jira/browse/OOZIE-2231
> Project: Oozie
>  Issue Type: Bug
>  Components: HA
>Reporter: Purshotam Shah
>Assignee: Abhishek Bafna
>Priority: Blocker
> Fix For: 5.0.0
>
> Attachments: OOZIE-2231-00.patch, OOZIE-2231-01.patch, 
> OOZIE-2231-02.patch, OOZIE-2231-02.patch, OOZIE-2231-03.patch
>
>
> It have some fix related to InterProcessReadWriteLock, ChildReaper, 
> LeaderSelector which we use.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OOZIE-2458) 'oozie-setup.sh sharelib create' should ensure uploaded jars are world readable

[Abhishek Bafna (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-2458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Abhishek Bafna updated OOZIE-2458:
--
Fix Version/s: (was: 4.3.0)
   5.0.0

> 'oozie-setup.sh sharelib create' should ensure uploaded jars are world 
> readable
> ---
>
> Key: OOZIE-2458
> URL: https://issues.apache.org/jira/browse/OOZIE-2458
> Project: Oozie
>  Issue Type: Bug
>  Components: scripts
>Affects Versions: 4.1.0
>Reporter: Ferenc Denes
>Assignee: Ferenc Denes
>Priority: Critical
> Fix For: 5.0.0
>
> Attachments: OOZIE-2458-1.patch
>
>
> If the default umask of HDFS does not include world read then the Oozie 
> sharelib gets created accordingly, and then the jars can not be read at 
> runtime because Oozie workflows impersonate the submitting user.
> The 'oozie-setup.sh sharelib create' command should ensure that the installed 
> jars are world readable.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OOZIE-2259) Create a callback action

[Abhishek Bafna (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-2259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Abhishek Bafna updated OOZIE-2259:
--
Fix Version/s: (was: 4.3.0)
   5.0.0

> Create a callback action 
> -
>
> Key: OOZIE-2259
> URL: https://issues.apache.org/jira/browse/OOZIE-2259
> Project: Oozie
>  Issue Type: New Feature
>  Components: action
>Reporter: Jaydeep Vishwakarma
>Assignee: Jaydeep Vishwakarma
> Fix For: 5.0.0
>
> Attachments: OOZIE-2259-v1.patch, OOZIE-2259-v3.patch, 
> OOZIE-2259-v4.patch, OOZIE-2259-v5.patch, OOZIE-2259-v8.patch, 
> OOZIE-2259-v9.patch, OOZIE-2259_v6.patch, OOZIE-2259_v7.patch
>
>
> Need an action to send notification to external server by oozie. We should be 
> able to do multiple types of callback, Currently I know jms and http call. It 
> should suppose to have capability to call diffrent types of methods along 
> with n number of arguments. 
> The sample workflow with callback action 
> {code:xml}
> 
> ...
> 
> 
>   [HOST]
>   [METHOD]
>   
>   [KEY][VALUE]
>   
> ...
> 
> ...
> 
> ...
> 
> {code}
> HOST : by the host system can figure out if it is http or jms callback 
> action. System will send the notification to that host.
> METHOD : it can be POST/GET/QUEUE/TOPIC



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OOZIE-2536) Hadoop's cleanup of local directory in uber mode causing failures

[Abhishek Bafna (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-2536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Abhishek Bafna updated OOZIE-2536:
--
Fix Version/s: (was: 4.3.0)
   5.0.0

> Hadoop's cleanup of local directory in uber mode causing failures
> -
>
> Key: OOZIE-2536
> URL: https://issues.apache.org/jira/browse/OOZIE-2536
> Project: Oozie
>  Issue Type: Bug
>Reporter: Satish Subhashrao Saley
>Assignee: Satish Subhashrao Saley
>Priority: Blocker
> Fix For: 5.0.0
>
>
> In out environment, we faced an issue where uberized Shell action was getting 
> stuck even though the shell action got completed with status 0. Please refer 
> the attached syslog and stdout if launcher job, here I point out partially
> stdout :
> {quote}
> >>> Invoking Shell command line now >>
> Stdoutput myshellType=qmyshellUpdate
> Exit code of the Shell command 0
> <<< Invocation of Shell command completed <<<
> <<< Invocation of Main class completed <<<
> {quote} 
> syslog
> {quote}
> 2016-05-23 11:15:52,587 WARN [uber-SubtaskRunner] 
> org.apache.hadoop.mapred.LocalContainerLauncher: Unable to delete unexpected 
> local file/dir .action.xml.crc: insufficient permissions?
> 2016-05-23 11:15:52,588 FATAL [AsyncDispatcher event handler] 
> org.apache.hadoop.conf.Configuration: error parsing conf propagation-conf.xml
> java.io.FileNotFoundException: 
> /tmp/yarn-local/usercache/saley/appcache/application_1234_123/container_e01_1234_123_01_01/propagation-conf.xml
>  (No such file or directory)
> at java.io.FileInputStream.open0(Native Method)
> at java.io.FileInputStream.open(FileInputStream.java:195)
> at java.io.FileInputStream.(FileInputStream.java:138)
> at java.io.FileInputStream.(FileInputStream.java:93)
> at 
> sun.net.www.protocol.file.FileURLConnection.connect(FileURLConnection.java:90)
> at 
> sun.net.www.protocol.file.FileURLConnection.getInputStream(FileURLConnection.java:188)
> at java.net.URL.openStream(URL.java:1038)
> at org.apache.hadoop.conf.Configuration.parse(Configuration.java:2468)
> at 
> org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2539)
> at 
> org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2492)
> at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2405)
> at org.apache.hadoop.conf.Configuration.get(Configuration.java:981)
> at 
> org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:1031)
> at org.apache.hadoop.conf.Configuration.getInt(Configuration.java:1251)
> at 
> org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl.getMemoryRequired(TaskAttemptImpl.java:568)
> at 
> org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl.updateMillisCounters(TaskAttemptImpl.java:1295)
> at 
> org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl.createJobCounterUpdateEventTASucceeded(TaskAttemptImpl.java:1323)
> at 
> org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl.access$3500(TaskAttemptImpl.java:147)
> at 
> org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl$SucceededTransition.transition(TaskAttemptImpl.java:1710)
> at 
> org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl$SucceededTransition.transition(TaskAttemptImpl.java:1701)
> at 
> org.apache.hadoop.yarn.state.StateMachineFactory$SingleInternalArc.doTransition(StateMachineFactory.java:362)
> at 
> org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:302)
> at 
> org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:46)
> at 
> org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:448)
> at 
> org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl.handle(TaskAttemptImpl.java:1085)
> at 
> org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl.handle(TaskAttemptImpl.java:146)
> at 
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster$TaskAttemptEventDispatcher.handle(MRAppMaster.java:1394)
> at 
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster$TaskAttemptEventDispatcher.handle(MRAppMaster.java:1386)
> at 
> org.apache.hadoop.yarn.event.AsyncDispatcher.dispatch(AsyncDispatcher.java:184)
> at 
> org.apache.hadoop.yarn.event.AsyncDispatcher$1.run(AsyncDispatcher.java:110)
> at java.lang.Thread.run(Thread.java:745)
> 2016-05-23 11:15:52,590 FATAL [AsyncDispatcher event handler] 
> org.apache.hadoop.yarn.event.AsyncDispatcher: Error in dispatcher thread
> java.lang.RuntimeException: java.io.FileNotFoundException: 
> /grid/5/tmp/yarn-local/usercache/saley/appcache/application_1234_123/container_e01_1234_123_01_01/propagation-conf.xml
>  (No such file or direc

[jira] [Commented] (OOZIE-2613) Upgrade hive version from 0.13.1 to 1.2.0

[Abhishek Bafna (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15564345#comment-15564345
 ] 

Abhishek Bafna commented on OOZIE-2613:
---

Thanks [~rkanter]. Uploading the patch soon.

> Upgrade hive version from 0.13.1 to 1.2.0
> -
>
> Key: OOZIE-2613
> URL: https://issues.apache.org/jira/browse/OOZIE-2613
> Project: Oozie
>  Issue Type: Bug
>  Components: core
>Reporter: Abhishek Bafna
>Assignee: Abhishek Bafna
>Priority: Blocker
> Fix For: 4.3.0
>
> Attachments: OOZIE-2613-00.patch, OOZIE-2613-01.patch, 
> OOZIE-2613-02.patch, OOZIE-2613-03.patch, OOZIE-2613-05.patch, 
> OOZIE-2613-06.patch, OOZIE-2613-07.patch, OOZIE-2613-08.patch, 
> OOZIE-2613-09.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OOZIE-2613) Upgrade hive version from 0.13.1 to 1.2.0

[Abhishek Bafna (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-2613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Abhishek Bafna updated OOZIE-2613:
--
Attachment: OOZIE-2613-10.patch

> Upgrade hive version from 0.13.1 to 1.2.0
> -
>
> Key: OOZIE-2613
> URL: https://issues.apache.org/jira/browse/OOZIE-2613
> Project: Oozie
>  Issue Type: Bug
>  Components: core
>Reporter: Abhishek Bafna
>Assignee: Abhishek Bafna
>Priority: Blocker
> Fix For: 4.3.0
>
> Attachments: OOZIE-2613-00.patch, OOZIE-2613-01.patch, 
> OOZIE-2613-02.patch, OOZIE-2613-03.patch, OOZIE-2613-05.patch, 
> OOZIE-2613-06.patch, OOZIE-2613-07.patch, OOZIE-2613-08.patch, 
> OOZIE-2613-09.patch, OOZIE-2613-10.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2225) Add wild card filter for gathering jobs

[Abhishek Bafna (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15567460#comment-15567460
 ] 

Abhishek Bafna commented on OOZIE-2225:
---

Can you add the updated patch into Jira as well. Thanks. (I believe you have 
updated the review board only.)

> Add wild card filter for gathering jobs
> ---
>
> Key: OOZIE-2225
> URL: https://issues.apache.org/jira/browse/OOZIE-2225
> Project: Oozie
>  Issue Type: Improvement
>  Components: core
>Affects Versions: 4.1.0
>Reporter: Sai Chirravuri
>Assignee: Sai Chirravuri
> Attachments: OOZIE-2225-001.patch, OOZIE-2225-002.patch, 
> OOZIE-2225-003.patch, OOZIE-2225-004.patch, OOZIE-2225-005.patch
>
>
> Oozie currently supports name, user, group, status, frequency and unit as 
> filter params.
> Add wild card param that matches partially with name/user or complete match 
> with ID



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (OOZIE-2710) Oozie HCatalog example workflow fails

[Abhishek Bafna (JIRA)]

Abhishek Bafna created OOZIE-2710:
-

 Summary: Oozie HCatalog example workflow fails
 Key: OOZIE-2710
 URL: https://issues.apache.org/jira/browse/OOZIE-2710
 Project: Oozie
  Issue Type: Bug
  Components: core
Reporter: Abhishek Bafna
Assignee: Abhishek Bafna
Priority: Blocker
 Fix For: 4.3.0


Oozie HCatalog example workflow fails with the below exception:

{noformat}
2016-10-12 10:33:37,260 [uber-SubtaskRunner] ERROR org.apache.pig.PigServer  - 
exception during parsing: Error during parsing. Could not resolve 
org.apache.hive.hcatalog.pig.HCatLoader using imports: [, java.lang., 
org.apache.pig.builtin., org.apache.pig.impl.builtin.]
Failed to parse: Pig script failed to parse: 
 pig script failed to validate: 
org.apache.pig.backend.executionengine.ExecException: ERROR 1070: Could not 
resolve org.apache.hive.hcatalog.pig.HCatLoader using imports: [, java.lang., 
org.apache.pig.builtin., org.apache.pig.impl.builtin.]
at 
org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:199)
at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1819)
at org.apache.pig.PigServer$Graph.access$000(PigServer.java:1527)
at org.apache.pig.PigServer.parseAndBuild(PigServer.java:460)
at org.apache.pig.PigServer.executeBatch(PigServer.java:485)
at org.apache.pig.PigServer.executeBatch(PigServer.java:471)
at 
org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java:172)
at 
org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:235)
at 
org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:206)
at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:81)
at org.apache.pig.Main.run(Main.java:501)
at org.apache.pig.PigRunner.run(PigRunner.java:49)
at org.apache.oozie.action.hadoop.PigMain.runPigJob(PigMain.java:290)
at org.apache.oozie.action.hadoop.PigMain.run(PigMain.java:233)
at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:60)
at org.apache.oozie.action.hadoop.PigMain.main(PigMain.java:73)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at 
org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:232)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:453)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
at 
org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.runSubtask(LocalContainerLauncher.java:380)
at 
org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.runTask(LocalContainerLauncher.java:301)
at 
org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.access$200(LocalContainerLauncher.java:187)
at 
org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler$1.run(LocalContainerLauncher.java:230)
at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: 
 pig script failed to validate: 
org.apache.pig.backend.executionengine.ExecException: ERROR 1070: Could not 
resolve org.apache.hive.hcatalog.pig.HCatLoader using imports: [, java.lang., 
org.apache.pig.builtin., org.apache.pig.impl.builtin.]
at 
org.apache.pig.parser.LogicalPlanBuilder.validateFuncSpec(LogicalPlanBuilder.java:1339)
at 
org.apache.pig.parser.LogicalPlanBuilder.buildFuncSpec(LogicalPlanBuilder.java:1324)
at 
org.apache.pig.parser.LogicalPlanGenerator.func_clause(LogicalPlanGenerator.java:5184)
at 
org.apache.pig.parser.LogicalPlanGenerator.load_clause(LogicalPlanGenerator.java:3515)
at 
org.apache.pig.parser.LogicalPlanGenerator.op_clause(LogicalPlanGenerator.java:1625)
at 
org.apache.pig.parser.LogicalPlanGenerator.general_statement(LogicalPlanGenerator.java:1102)
at 
org.apache.pig.parser.LogicalPlanGenerator.statement(LogicalPlanGenerator.java:560)
at 
org.apache.pig.parser.LogicalPlanGenerator.query(LogicalPlanGenerator.java:421)
at 
org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:191)
... 32 more
Caused by: org.apache.pig.backend.executionengine.ExecException: ERROR 1070: 
Could not resolve org.apache.hive.hca

[jira] [Commented] (OOZIE-2710) Oozie HCatalog example workflow fails

[Abhishek Bafna (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15568570#comment-15568570
 ] 

Abhishek Bafna commented on OOZIE-2710:
---

Checked all the jars in {{pig}}, {{hive}} and {{hcatalog}}, none of them 
contains the {{HCatLoader}} class. Command used is:

{noformat}
for i in *.jar; do jar -tvf "$i" | grep -Hsi HCatLoader && echo "$i"; done
{noformat}

The class is available in the {{hive-hcatalog-pig-adapter}} module. Added the 
same into oozie pig share lib module, excluded some of the hive-shims and 
curator jars.

> Oozie HCatalog example workflow fails
> -
>
> Key: OOZIE-2710
> URL: https://issues.apache.org/jira/browse/OOZIE-2710
> Project: Oozie
>  Issue Type: Bug
>  Components: core
>Reporter: Abhishek Bafna
>Assignee: Abhishek Bafna
>Priority: Blocker
> Fix For: 4.3.0
>
>
> Oozie HCatalog example workflow fails with the below exception:
> {noformat}
> 2016-10-12 10:33:37,260 [uber-SubtaskRunner] ERROR org.apache.pig.PigServer  
> - exception during parsing: Error during parsing. Could not resolve 
> org.apache.hive.hcatalog.pig.HCatLoader using imports: [, java.lang., 
> org.apache.pig.builtin., org.apache.pig.impl.builtin.]
> Failed to parse: Pig script failed to parse: 
>  pig script failed to validate: 
> org.apache.pig.backend.executionengine.ExecException: ERROR 1070: Could not 
> resolve org.apache.hive.hcatalog.pig.HCatLoader using imports: [, java.lang., 
> org.apache.pig.builtin., org.apache.pig.impl.builtin.]
>   at 
> org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:199)
>   at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1819)
>   at org.apache.pig.PigServer$Graph.access$000(PigServer.java:1527)
>   at org.apache.pig.PigServer.parseAndBuild(PigServer.java:460)
>   at org.apache.pig.PigServer.executeBatch(PigServer.java:485)
>   at org.apache.pig.PigServer.executeBatch(PigServer.java:471)
>   at 
> org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java:172)
>   at 
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:235)
>   at 
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:206)
>   at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:81)
>   at org.apache.pig.Main.run(Main.java:501)
>   at org.apache.pig.PigRunner.run(PigRunner.java:49)
>   at org.apache.oozie.action.hadoop.PigMain.runPigJob(PigMain.java:290)
>   at org.apache.oozie.action.hadoop.PigMain.run(PigMain.java:233)
>   at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:60)
>   at org.apache.oozie.action.hadoop.PigMain.main(PigMain.java:73)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:497)
>   at 
> org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:232)
>   at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
>   at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:453)
>   at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
>   at 
> org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.runSubtask(LocalContainerLauncher.java:380)
>   at 
> org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.runTask(LocalContainerLauncher.java:301)
>   at 
> org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.access$200(LocalContainerLauncher.java:187)
>   at 
> org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler$1.run(LocalContainerLauncher.java:230)
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>   at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>   at java.lang.Thread.run(Thread.java:745)
> Caused by: 
>  pig script failed to validate: 
> org.apache.pig.backend.executionengine.ExecException: ERROR 1070: Could not 
> resolve org.apache.hive.hcatalog.pig.HCatLoader using imports: [, java.lang., 
> org.apache.pig.builtin., org.apache.pig.impl.builtin.]
>   at 
> org.apache.pig.parser.LogicalPlanBuilder.validateFuncSpec(LogicalPlanBuilder.java:1339)
>   at 
> org.apache.pig.parser.LogicalPlanBuilder.buildFuncSpec(LogicalPlanBuilder.java:1324)
>   at 
> org.apache.pig.parser.LogicalPlanGenerator.func_clause(LogicalPlanGenerator.java:5184)
>   at 
> org.apache.pig.parser.

[jira] [Updated] (OOZIE-2710) Oozie HCatalog example workflow fails

[Abhishek Bafna (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-2710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Abhishek Bafna updated OOZIE-2710:
--
Attachment: OOZIE-2710-00.patch

> Oozie HCatalog example workflow fails
> -
>
> Key: OOZIE-2710
> URL: https://issues.apache.org/jira/browse/OOZIE-2710
> Project: Oozie
>  Issue Type: Bug
>  Components: core
>Reporter: Abhishek Bafna
>Assignee: Abhishek Bafna
>Priority: Blocker
> Fix For: 4.3.0
>
> Attachments: OOZIE-2710-00.patch
>
>
> Oozie HCatalog example workflow fails with the below exception:
> {noformat}
> 2016-10-12 10:33:37,260 [uber-SubtaskRunner] ERROR org.apache.pig.PigServer  
> - exception during parsing: Error during parsing. Could not resolve 
> org.apache.hive.hcatalog.pig.HCatLoader using imports: [, java.lang., 
> org.apache.pig.builtin., org.apache.pig.impl.builtin.]
> Failed to parse: Pig script failed to parse: 
>  pig script failed to validate: 
> org.apache.pig.backend.executionengine.ExecException: ERROR 1070: Could not 
> resolve org.apache.hive.hcatalog.pig.HCatLoader using imports: [, java.lang., 
> org.apache.pig.builtin., org.apache.pig.impl.builtin.]
>   at 
> org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:199)
>   at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1819)
>   at org.apache.pig.PigServer$Graph.access$000(PigServer.java:1527)
>   at org.apache.pig.PigServer.parseAndBuild(PigServer.java:460)
>   at org.apache.pig.PigServer.executeBatch(PigServer.java:485)
>   at org.apache.pig.PigServer.executeBatch(PigServer.java:471)
>   at 
> org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java:172)
>   at 
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:235)
>   at 
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:206)
>   at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:81)
>   at org.apache.pig.Main.run(Main.java:501)
>   at org.apache.pig.PigRunner.run(PigRunner.java:49)
>   at org.apache.oozie.action.hadoop.PigMain.runPigJob(PigMain.java:290)
>   at org.apache.oozie.action.hadoop.PigMain.run(PigMain.java:233)
>   at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:60)
>   at org.apache.oozie.action.hadoop.PigMain.main(PigMain.java:73)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:497)
>   at 
> org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:232)
>   at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
>   at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:453)
>   at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
>   at 
> org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.runSubtask(LocalContainerLauncher.java:380)
>   at 
> org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.runTask(LocalContainerLauncher.java:301)
>   at 
> org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.access$200(LocalContainerLauncher.java:187)
>   at 
> org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler$1.run(LocalContainerLauncher.java:230)
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>   at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>   at java.lang.Thread.run(Thread.java:745)
> Caused by: 
>  pig script failed to validate: 
> org.apache.pig.backend.executionengine.ExecException: ERROR 1070: Could not 
> resolve org.apache.hive.hcatalog.pig.HCatLoader using imports: [, java.lang., 
> org.apache.pig.builtin., org.apache.pig.impl.builtin.]
>   at 
> org.apache.pig.parser.LogicalPlanBuilder.validateFuncSpec(LogicalPlanBuilder.java:1339)
>   at 
> org.apache.pig.parser.LogicalPlanBuilder.buildFuncSpec(LogicalPlanBuilder.java:1324)
>   at 
> org.apache.pig.parser.LogicalPlanGenerator.func_clause(LogicalPlanGenerator.java:5184)
>   at 
> org.apache.pig.parser.LogicalPlanGenerator.load_clause(LogicalPlanGenerator.java:3515)
>   at 
> org.apache.pig.parser.LogicalPlanGenerator.op_clause(LogicalPlanGenerator.java:1625)
>   at 
> org.apache.pig.parser.LogicalPlanGenerator.general_statement(LogicalPlanGenerator.java:1102)
>   at 
> org.apache.pig.parser.LogicalPlanGenerator.statement(LogicalPlanGenerator.java:560)
>

[jira] [Commented] (OOZIE-2710) Oozie HCatalog example workflow fails

[Abhishek Bafna (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15572036#comment-15572036
 ] 

Abhishek Bafna commented on OOZIE-2710:
---

Ping: [~rkanter], [~rohini].

> Oozie HCatalog example workflow fails
> -
>
> Key: OOZIE-2710
> URL: https://issues.apache.org/jira/browse/OOZIE-2710
> Project: Oozie
>  Issue Type: Bug
>  Components: core
>Reporter: Abhishek Bafna
>Assignee: Abhishek Bafna
>Priority: Blocker
> Fix For: 4.3.0
>
> Attachments: OOZIE-2710-00.patch
>
>
> Oozie HCatalog example workflow fails with the below exception:
> {noformat}
> 2016-10-12 10:33:37,260 [uber-SubtaskRunner] ERROR org.apache.pig.PigServer  
> - exception during parsing: Error during parsing. Could not resolve 
> org.apache.hive.hcatalog.pig.HCatLoader using imports: [, java.lang., 
> org.apache.pig.builtin., org.apache.pig.impl.builtin.]
> Failed to parse: Pig script failed to parse: 
>  pig script failed to validate: 
> org.apache.pig.backend.executionengine.ExecException: ERROR 1070: Could not 
> resolve org.apache.hive.hcatalog.pig.HCatLoader using imports: [, java.lang., 
> org.apache.pig.builtin., org.apache.pig.impl.builtin.]
>   at 
> org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:199)
>   at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1819)
>   at org.apache.pig.PigServer$Graph.access$000(PigServer.java:1527)
>   at org.apache.pig.PigServer.parseAndBuild(PigServer.java:460)
>   at org.apache.pig.PigServer.executeBatch(PigServer.java:485)
>   at org.apache.pig.PigServer.executeBatch(PigServer.java:471)
>   at 
> org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java:172)
>   at 
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:235)
>   at 
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:206)
>   at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:81)
>   at org.apache.pig.Main.run(Main.java:501)
>   at org.apache.pig.PigRunner.run(PigRunner.java:49)
>   at org.apache.oozie.action.hadoop.PigMain.runPigJob(PigMain.java:290)
>   at org.apache.oozie.action.hadoop.PigMain.run(PigMain.java:233)
>   at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:60)
>   at org.apache.oozie.action.hadoop.PigMain.main(PigMain.java:73)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:497)
>   at 
> org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:232)
>   at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
>   at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:453)
>   at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
>   at 
> org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.runSubtask(LocalContainerLauncher.java:380)
>   at 
> org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.runTask(LocalContainerLauncher.java:301)
>   at 
> org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.access$200(LocalContainerLauncher.java:187)
>   at 
> org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler$1.run(LocalContainerLauncher.java:230)
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>   at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>   at java.lang.Thread.run(Thread.java:745)
> Caused by: 
>  pig script failed to validate: 
> org.apache.pig.backend.executionengine.ExecException: ERROR 1070: Could not 
> resolve org.apache.hive.hcatalog.pig.HCatLoader using imports: [, java.lang., 
> org.apache.pig.builtin., org.apache.pig.impl.builtin.]
>   at 
> org.apache.pig.parser.LogicalPlanBuilder.validateFuncSpec(LogicalPlanBuilder.java:1339)
>   at 
> org.apache.pig.parser.LogicalPlanBuilder.buildFuncSpec(LogicalPlanBuilder.java:1324)
>   at 
> org.apache.pig.parser.LogicalPlanGenerator.func_clause(LogicalPlanGenerator.java:5184)
>   at 
> org.apache.pig.parser.LogicalPlanGenerator.load_clause(LogicalPlanGenerator.java:3515)
>   at 
> org.apache.pig.parser.LogicalPlanGenerator.op_clause(LogicalPlanGenerator.java:1625)
>   at 
> org.apache.pig.parser.LogicalPlanGenerator.general_statement(LogicalPlanGenerator.java:1102)
>   at 
> org.apache.pig.parser.LogicalPlanG

[jira] [Commented] (OOZIE-2621) Use hive-exec--core instead of hive-exec in oozie-core

[Abhishek Bafna (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15577185#comment-15577185
 ] 

Abhishek Bafna commented on OOZIE-2621:
---

[~rkanter], sure, we can definitely include this into 4.3. Thanks.

> Use hive-exec--core instead of hive-exec in oozie-core
> ---
>
> Key: OOZIE-2621
> URL: https://issues.apache.org/jira/browse/OOZIE-2621
> Project: Oozie
>  Issue Type: Bug
>Reporter: Peter Cseh
>Assignee: Peter Cseh
> Attachments: OOZIE-2621.001.patch
>
>
> hive-exec.jar contains the dependencies of Hive and they can cause issues on 
> the classpath: HIVE-1640, HIVE-5733
> We've ran into and issue when Oozie won't start because of different Jackson 
> versions.
> We should use hive-exec-core.jar (HIVE-7423) instead of hive-exec.jar which 
> is a thin version of hive-exec.jar 
> Pig is already using hive-exec-core: 
> https://github.com/apache/pig/blob/spark/ivy.xml#L423



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2710) Oozie HCatalog example workflow fails

[Abhishek Bafna (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15577188#comment-15577188
 ] 

Abhishek Bafna commented on OOZIE-2710:
---

[~rkanter] Can you please take look at this issue? I faced it while executing 
the oozie-examples with 4.3 build. Thanks.

> Oozie HCatalog example workflow fails
> -
>
> Key: OOZIE-2710
> URL: https://issues.apache.org/jira/browse/OOZIE-2710
> Project: Oozie
>  Issue Type: Bug
>  Components: core
>Reporter: Abhishek Bafna
>Assignee: Abhishek Bafna
>Priority: Blocker
> Fix For: 4.3.0
>
> Attachments: OOZIE-2710-00.patch
>
>
> Oozie HCatalog example workflow fails with the below exception:
> {noformat}
> 2016-10-12 10:33:37,260 [uber-SubtaskRunner] ERROR org.apache.pig.PigServer  
> - exception during parsing: Error during parsing. Could not resolve 
> org.apache.hive.hcatalog.pig.HCatLoader using imports: [, java.lang., 
> org.apache.pig.builtin., org.apache.pig.impl.builtin.]
> Failed to parse: Pig script failed to parse: 
>  pig script failed to validate: 
> org.apache.pig.backend.executionengine.ExecException: ERROR 1070: Could not 
> resolve org.apache.hive.hcatalog.pig.HCatLoader using imports: [, java.lang., 
> org.apache.pig.builtin., org.apache.pig.impl.builtin.]
>   at 
> org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:199)
>   at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1819)
>   at org.apache.pig.PigServer$Graph.access$000(PigServer.java:1527)
>   at org.apache.pig.PigServer.parseAndBuild(PigServer.java:460)
>   at org.apache.pig.PigServer.executeBatch(PigServer.java:485)
>   at org.apache.pig.PigServer.executeBatch(PigServer.java:471)
>   at 
> org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java:172)
>   at 
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:235)
>   at 
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:206)
>   at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:81)
>   at org.apache.pig.Main.run(Main.java:501)
>   at org.apache.pig.PigRunner.run(PigRunner.java:49)
>   at org.apache.oozie.action.hadoop.PigMain.runPigJob(PigMain.java:290)
>   at org.apache.oozie.action.hadoop.PigMain.run(PigMain.java:233)
>   at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:60)
>   at org.apache.oozie.action.hadoop.PigMain.main(PigMain.java:73)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:497)
>   at 
> org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:232)
>   at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
>   at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:453)
>   at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
>   at 
> org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.runSubtask(LocalContainerLauncher.java:380)
>   at 
> org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.runTask(LocalContainerLauncher.java:301)
>   at 
> org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.access$200(LocalContainerLauncher.java:187)
>   at 
> org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler$1.run(LocalContainerLauncher.java:230)
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>   at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>   at java.lang.Thread.run(Thread.java:745)
> Caused by: 
>  pig script failed to validate: 
> org.apache.pig.backend.executionengine.ExecException: ERROR 1070: Could not 
> resolve org.apache.hive.hcatalog.pig.HCatLoader using imports: [, java.lang., 
> org.apache.pig.builtin., org.apache.pig.impl.builtin.]
>   at 
> org.apache.pig.parser.LogicalPlanBuilder.validateFuncSpec(LogicalPlanBuilder.java:1339)
>   at 
> org.apache.pig.parser.LogicalPlanBuilder.buildFuncSpec(LogicalPlanBuilder.java:1324)
>   at 
> org.apache.pig.parser.LogicalPlanGenerator.func_clause(LogicalPlanGenerator.java:5184)
>   at 
> org.apache.pig.parser.LogicalPlanGenerator.load_clause(LogicalPlanGenerator.java:3515)
>   at 
> org.apache.pig.parser.LogicalPlanGenerator.op_clause(LogicalPlanGenerator.java:1625)
>   at 
> org.apache.pig.parser.LogicalPlanGenerator.gener

[jira] [Commented] (OOZIE-2621) Use hive-exec--core instead of hive-exec in oozie-core

[Abhishek Bafna (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15580093#comment-15580093
 ] 

Abhishek Bafna commented on OOZIE-2621:
---

[~gezapeti] There is one more exception while running locally, but goes pass 
through that.

{noformat}
18:53:34,099  WARN ShellBasedUnixGroupsMapping:86 - got exception trying to get 
groups for user foo
org.apache.hadoop.util.Shell$ExitCodeException: id: foo: no such user

at org.apache.hadoop.util.Shell.runCommand(Shell.java:505)
at org.apache.hadoop.util.Shell.run(Shell.java:418)
at 
org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:650)
at org.apache.hadoop.util.Shell.execCommand(Shell.java:739)
at org.apache.hadoop.util.Shell.execCommand(Shell.java:722)
at 
org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getUnixGroups(ShellBasedUnixGroupsMapping.java:83)
at 
org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getGroups(ShellBasedUnixGroupsMapping.java:52)
at 
org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback.getGroups(JniBasedUnixGroupsMappingWithFallback.java:50)
at org.apache.hadoop.security.Groups.getGroups(Groups.java:139)
at 
org.apache.hadoop.security.UserGroupInformation$TestingGroups.getGroups(UserGroupInformation.java:1222)
at 
org.apache.hadoop.security.UserGroupInformation.getGroupNames(UserGroupInformation.java:1409)
at 
org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.(FSPermissionChecker.java:81)
at 
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getPermissionChecker(FSNamesystem.java:3306)
at 
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getFileInfo(FSNamesystem.java:3487)
at 
org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getFileInfo(NameNodeRpcServer.java:779)
at 
org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getFileInfo(ClientNamenodeProtocolServerSideTranslatorPB.java:764)
at 
org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
at 
org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:585)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:928)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2013)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2009)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2007)
18:53:34,102  WARN UserGroupInformation:1412 - No groups available for user foo
18:53:34,102  INFO audit:7743 - allowed=trueugi=foo (auth:SIMPLE)   
ip=/127.0.0.1   cmd=getfileinfo src=/tmp/hive   dst=nullperm=null
{noformat}

Also, OOZIE-2613 introduced the below change for the {{hadoop-1}} profile. 
{{MiniHCatServer#resetDefaultDBCreation}} got removed.

{code}
buffer.append("CREATE DATABASE IF NOT EXISTS default;").append(NEW_LINE);
{code}

though I tried running the testcases after removing this line, but it still 
fails. (Did not see any exception but these logs)

{noformat}
20:22:29,898  WARN ObjectStore:568 - Failed to get database default, returning 
NoSuchObjectException
...
20:22:43,014  INFO Hive2ActionExecutor:520 - USER[test] GROUP[-] TOKEN[] 
APP[testApp] JOB[000-161016202228110-oozie-abaf-W] 
ACTION[000-161016202228110-oozie-abaf-W@hive2-action] action completed, 
external ID [job_1476629530322_0001]
20:22:43,051  WARN Hive2ActionExecutor:523 - USER[test] GROUP[-] TOKEN[] 
APP[testApp] JOB[000-161016202228110-oozie-abaf-W] 
ACTION[000-161016202228110-oozie-abaf-W@hive2-action] Launcher ERROR, 
reason: Main class [org.apache.oozie.action.hadoop.Hive2Main], exit code [2]
20:22:43,053  INFO Services:520 - USER[test] GROUP[-] TOKEN[] APP[testApp] 
JOB[000-161016202228110-oozie-abaf-W] 
ACTION[000-161016202228110-oozie-abaf-W@hive2-action] Shutdown
{noformat}

> Use hive-exec--core instead of hive-exec in oozie-core
> ---
>
> Key: OOZIE-2621
> URL: https://issues.apache.org/jira/browse/OOZIE-2621
> Project: Oozie
>  Issue Type: Bug
>Reporter: Peter Cseh
>Assignee: Peter Cseh
> Attachments: OOZIE-2621.001.patch
>
>
> hive-exec.jar contains the dependencies of Hive and they can cause issues on 
> the classpath: HIVE-1640, HIVE-5733
> We've ran into and issue when Oozie won't start because of different Jackson 
> versions.
> We should use hive-exec-core.jar (HIVE-7423) instead of hive-e

[jira] [Commented] (OOZIE-2621) Use hive-exec--core instead of hive-exec in oozie-core

[Abhishek Bafna (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15584940#comment-15584940
 ] 

Abhishek Bafna commented on OOZIE-2621:
---

[~gezapeti] I am also not able to re-produce the issue. I guess, we can submit 
the Jenkins job again or submit the same patch again. Thanks.

> Use hive-exec--core instead of hive-exec in oozie-core
> ---
>
> Key: OOZIE-2621
> URL: https://issues.apache.org/jira/browse/OOZIE-2621
> Project: Oozie
>  Issue Type: Bug
>Reporter: Peter Cseh
>Assignee: Peter Cseh
> Attachments: OOZIE-2621.001.patch, OOZIE-2621.002.patch
>
>
> hive-exec.jar contains the dependencies of Hive and they can cause issues on 
> the classpath: HIVE-1640, HIVE-5733
> We've ran into and issue when Oozie won't start because of different Jackson 
> versions.
> We should use hive-exec-core.jar (HIVE-7423) instead of hive-exec.jar which 
> is a thin version of hive-exec.jar 
> Pig is already using hive-exec-core: 
> https://github.com/apache/pig/blob/spark/ivy.xml#L423



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2621) Use hive-exec--core instead of hive-exec in oozie-core

[Abhishek Bafna (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15585092#comment-15585092
 ] 

Abhishek Bafna commented on OOZIE-2621:
---

[~gezapeti] It does not compile with {{hadoop-1}} profile.

{noformat}
[ERROR] Failed to execute goal on project oozie-core: Could not resolve 
dependencies for project org.apache.oozie:oozie-core:jar:4.4.0-SNAPSHOT: Could 
not find artifact org.apache.hive:hive-exec:jar:core:0.13.1 in public 
(http://nexus-private.hortonworks.com/nexus/content/groups/public/) -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e 
switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please 
read the following articles:
[ERROR] [Help 1] 
http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException
[ERROR] 
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn  -rf :oozie-core
{noformat}

> Use hive-exec--core instead of hive-exec in oozie-core
> ---
>
> Key: OOZIE-2621
> URL: https://issues.apache.org/jira/browse/OOZIE-2621
> Project: Oozie
>  Issue Type: Bug
>Reporter: Peter Cseh
>Assignee: Peter Cseh
> Attachments: OOZIE-2621.001.patch, OOZIE-2621.002.patch
>
>
> hive-exec.jar contains the dependencies of Hive and they can cause issues on 
> the classpath: HIVE-1640, HIVE-5733
> We've ran into and issue when Oozie won't start because of different Jackson 
> versions.
> We should use hive-exec-core.jar (HIVE-7423) instead of hive-exec.jar which 
> is a thin version of hive-exec.jar 
> Pig is already using hive-exec-core: 
> https://github.com/apache/pig/blob/spark/ivy.xml#L423



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2621) Use hive-exec--core instead of hive-exec in oozie-core

[Abhishek Bafna (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15587812#comment-15587812
 ] 

Abhishek Bafna commented on OOZIE-2621:
---

LGTM +1 non-binding.

> Use hive-exec--core instead of hive-exec in oozie-core
> ---
>
> Key: OOZIE-2621
> URL: https://issues.apache.org/jira/browse/OOZIE-2621
> Project: Oozie
>  Issue Type: Bug
>Reporter: Peter Cseh
>Assignee: Peter Cseh
> Attachments: OOZIE-2621.001.patch, OOZIE-2621.002.patch, 
> OOZIE-2621.003.patch
>
>
> hive-exec.jar contains the dependencies of Hive and they can cause issues on 
> the classpath: HIVE-1640, HIVE-5733
> We've ran into and issue when Oozie won't start because of different Jackson 
> versions.
> We should use hive-exec-core.jar (HIVE-7423) instead of hive-exec.jar which 
> is a thin version of hive-exec.jar 
> Pig is already using hive-exec-core: 
> https://github.com/apache/pig/blob/spark/ivy.xml#L423



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2710) Oozie HCatalog example workflow fails

[Abhishek Bafna (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15587819#comment-15587819
 ] 

Abhishek Bafna commented on OOZIE-2710:
---

Thank you [~gezapeti] for the review.
Ping: [~rkanter]

> Oozie HCatalog example workflow fails
> -
>
> Key: OOZIE-2710
> URL: https://issues.apache.org/jira/browse/OOZIE-2710
> Project: Oozie
>  Issue Type: Bug
>  Components: core
>Reporter: Abhishek Bafna
>Assignee: Abhishek Bafna
>Priority: Blocker
> Fix For: 4.3.0
>
> Attachments: OOZIE-2710-00.patch
>
>
> Oozie HCatalog example workflow fails with the below exception:
> {noformat}
> 2016-10-12 10:33:37,260 [uber-SubtaskRunner] ERROR org.apache.pig.PigServer  
> - exception during parsing: Error during parsing. Could not resolve 
> org.apache.hive.hcatalog.pig.HCatLoader using imports: [, java.lang., 
> org.apache.pig.builtin., org.apache.pig.impl.builtin.]
> Failed to parse: Pig script failed to parse: 
>  pig script failed to validate: 
> org.apache.pig.backend.executionengine.ExecException: ERROR 1070: Could not 
> resolve org.apache.hive.hcatalog.pig.HCatLoader using imports: [, java.lang., 
> org.apache.pig.builtin., org.apache.pig.impl.builtin.]
>   at 
> org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:199)
>   at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1819)
>   at org.apache.pig.PigServer$Graph.access$000(PigServer.java:1527)
>   at org.apache.pig.PigServer.parseAndBuild(PigServer.java:460)
>   at org.apache.pig.PigServer.executeBatch(PigServer.java:485)
>   at org.apache.pig.PigServer.executeBatch(PigServer.java:471)
>   at 
> org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java:172)
>   at 
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:235)
>   at 
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:206)
>   at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:81)
>   at org.apache.pig.Main.run(Main.java:501)
>   at org.apache.pig.PigRunner.run(PigRunner.java:49)
>   at org.apache.oozie.action.hadoop.PigMain.runPigJob(PigMain.java:290)
>   at org.apache.oozie.action.hadoop.PigMain.run(PigMain.java:233)
>   at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:60)
>   at org.apache.oozie.action.hadoop.PigMain.main(PigMain.java:73)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:497)
>   at 
> org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:232)
>   at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
>   at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:453)
>   at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
>   at 
> org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.runSubtask(LocalContainerLauncher.java:380)
>   at 
> org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.runTask(LocalContainerLauncher.java:301)
>   at 
> org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.access$200(LocalContainerLauncher.java:187)
>   at 
> org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler$1.run(LocalContainerLauncher.java:230)
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>   at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>   at java.lang.Thread.run(Thread.java:745)
> Caused by: 
>  pig script failed to validate: 
> org.apache.pig.backend.executionengine.ExecException: ERROR 1070: Could not 
> resolve org.apache.hive.hcatalog.pig.HCatLoader using imports: [, java.lang., 
> org.apache.pig.builtin., org.apache.pig.impl.builtin.]
>   at 
> org.apache.pig.parser.LogicalPlanBuilder.validateFuncSpec(LogicalPlanBuilder.java:1339)
>   at 
> org.apache.pig.parser.LogicalPlanBuilder.buildFuncSpec(LogicalPlanBuilder.java:1324)
>   at 
> org.apache.pig.parser.LogicalPlanGenerator.func_clause(LogicalPlanGenerator.java:5184)
>   at 
> org.apache.pig.parser.LogicalPlanGenerator.load_clause(LogicalPlanGenerator.java:3515)
>   at 
> org.apache.pig.parser.LogicalPlanGenerator.op_clause(LogicalPlanGenerator.java:1625)
>   at 
> org.apache.pig.parser.LogicalPlanGenerator.general_statement(LogicalPlanGenerator.java:1102)
>   at 
> org.apac

[jira] [Commented] (OOZIE-1986) Add FindBugs report to pre-commit build

[Abhishek Bafna (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-1986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15588676#comment-15588676
 ] 

Abhishek Bafna commented on OOZIE-1986:
---

I guess issue is because of the missing execute flag in file permissions. 
Please check. Thanks.

{noformat}
-rwxr-xr-x  1 abafna  staff   4292 Oct 10 17:38 test-patch-10-compile
-rw-r--r--  1 abafna  staff   9381 Oct 19 11:15 test-patch-11-findbugs-diff
-rw-r--r--  1 abafna  staff 33 Oct 19 11:15 
test-patch-11-findbugs-diff-0.1.0-all.jar.md5
-rwxr-xr-x  1 abafna  staff   3471 Jan 25  2016 
test-patch-15-backwards-compatibility
{noformat}

> Add FindBugs report to pre-commit build
> ---
>
> Key: OOZIE-1986
> URL: https://issues.apache.org/jira/browse/OOZIE-1986
> Project: Oozie
>  Issue Type: Bug
>  Components: tests
>Reporter: Robert Kanter
>Assignee: Andras Piros
> Fix For: 5.0.0
>
> Attachments: OOZIE-1986.001.patch, OOZIE-1986.002.patch, 
> OOZIE-1986.003.patch, OOZIE-1986.004.patch, OOZIE-1986.005.patch, 
> OOZIE-1986.006.patch, OOZIE-1986.007.patch, OOZIE-1986.008.patch, 
> OOZIE-1986.009.patch, OOZIE-1986.010.patch, summary.txt
>
>
> Now that we have OOZIE-1793, it will be really useful to have the pre-commit 
> build generate a findbugs report.  It should probably before and after the 
> patch and only complain it there are more after.  And it should link to the 
> report.  Ideally, this would do some kind of diff (like Hadoop does) so it's 
> easy to identify the new findbugs items.
> You can generate the findbugs reports by running:
> {noformat}
> mvn verify -DskipTests
> {noformat}
> This also runs checkstyle



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2712) Merge Spark opts with two possible options

[Abhishek Bafna (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15588771#comment-15588771
 ] 

Abhishek Bafna commented on OOZIE-2712:
---

{code}
propertiesToMerge.put(DRIVER_CLASSPATH_OPTION, DRIVER_CLASSPATH);
propertiesToMerge.put(DRIVER_MEMORY_OPTION, DRIVER_MEMORY);
propertiesToMerge.put(DRIVER_LIBRARY_PATH_OPTION, 
DRIVER_EXTRA_LIBRARY_PATH);
propertiesToMerge.put(NUM_EXECUTOR_OPTION, EXECUTOR_INSTANCES);
propertiesToMerge.put(DRIVER_JAVA_OPTIONS_OPTION, 
DRIVER_EXTRA_JAVA_OPTIONS);
propertySeparators.put(DRIVER_CLASSPATH, ",");
propertySeparators.put(DRIVER_EXTRA_LIBRARY_PATH, ",");
propertySeparators.put(DRIVER_EXTRA_JAVA_OPTIONS, "");
propertySeparators.put(EXECUTOR_INSTANCES, "");
propertySeparators.put(DRIVER_EXTRA_JAVA_OPTIONS, " ");
{code}

{{DRIVER_EXTRA_JAVA_OPTIONS}} is added twice into {{propertySeparators}} which 
is why {{DRIVER_MEMORY}} is missed out from the map.


> Merge Spark opts with two possible options 
> ---
>
> Key: OOZIE-2712
> URL: https://issues.apache.org/jira/browse/OOZIE-2712
> Project: Oozie
>  Issue Type: Bug
>Reporter: Peter Cseh
>Assignee: Peter Cseh
> Attachments: OOZIE-2712.001.patch
>
>
> Issues like OOZIE-2658 can happen with more properties.
> || --conf ||  overwritten by ||
> |spark.driver.memory | --driver-memory |
> |spark.driver.extraJavaOptions | --driver-java-options|
> |spark.driver.extraLibraryPath|--driver-library-path |
> |spark.executor.instances|--num-executors|
> |spark.driver.extraClassPath| --driver-class-path|
> We should make sure that one is converted to the other (the current pattern 
> is to use --conf versions of these) so we don't run into issues where the 
> log4j properties are overwritten. (OOZIE-2705 can be affected by this)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OOZIE-2621) Use hive-exec--core instead of hive-exec in oozie-core

[Abhishek Bafna (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-2621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Abhishek Bafna updated OOZIE-2621:
--
Fix Version/s: 4.3.0

> Use hive-exec--core instead of hive-exec in oozie-core
> ---
>
> Key: OOZIE-2621
> URL: https://issues.apache.org/jira/browse/OOZIE-2621
> Project: Oozie
>  Issue Type: Bug
>Reporter: Peter Cseh
>Assignee: Peter Cseh
> Fix For: 4.3.0
>
> Attachments: OOZIE-2621.001.patch, OOZIE-2621.002.patch, 
> OOZIE-2621.003.patch
>
>
> hive-exec.jar contains the dependencies of Hive and they can cause issues on 
> the classpath: HIVE-1640, HIVE-5733
> We've ran into and issue when Oozie won't start because of different Jackson 
> versions.
> We should use hive-exec-core.jar (HIVE-7423) instead of hive-exec.jar which 
> is a thin version of hive-exec.jar 
> Pig is already using hive-exec-core: 
> https://github.com/apache/pig/blob/spark/ivy.xml#L423



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OOZIE-2710) Oozie HCatalog example workflow fails

[Abhishek Bafna (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-2710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Abhishek Bafna updated OOZIE-2710:
--
Component/s: examples

> Oozie HCatalog example workflow fails
> -
>
> Key: OOZIE-2710
> URL: https://issues.apache.org/jira/browse/OOZIE-2710
> Project: Oozie
>  Issue Type: Bug
>  Components: core, examples
>Reporter: Abhishek Bafna
>Assignee: Abhishek Bafna
>Priority: Blocker
> Fix For: 4.3.0
>
> Attachments: OOZIE-2710-00.patch
>
>
> Oozie HCatalog example workflow fails with the below exception:
> {noformat}
> 2016-10-12 10:33:37,260 [uber-SubtaskRunner] ERROR org.apache.pig.PigServer  
> - exception during parsing: Error during parsing. Could not resolve 
> org.apache.hive.hcatalog.pig.HCatLoader using imports: [, java.lang., 
> org.apache.pig.builtin., org.apache.pig.impl.builtin.]
> Failed to parse: Pig script failed to parse: 
>  pig script failed to validate: 
> org.apache.pig.backend.executionengine.ExecException: ERROR 1070: Could not 
> resolve org.apache.hive.hcatalog.pig.HCatLoader using imports: [, java.lang., 
> org.apache.pig.builtin., org.apache.pig.impl.builtin.]
>   at 
> org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:199)
>   at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1819)
>   at org.apache.pig.PigServer$Graph.access$000(PigServer.java:1527)
>   at org.apache.pig.PigServer.parseAndBuild(PigServer.java:460)
>   at org.apache.pig.PigServer.executeBatch(PigServer.java:485)
>   at org.apache.pig.PigServer.executeBatch(PigServer.java:471)
>   at 
> org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java:172)
>   at 
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:235)
>   at 
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:206)
>   at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:81)
>   at org.apache.pig.Main.run(Main.java:501)
>   at org.apache.pig.PigRunner.run(PigRunner.java:49)
>   at org.apache.oozie.action.hadoop.PigMain.runPigJob(PigMain.java:290)
>   at org.apache.oozie.action.hadoop.PigMain.run(PigMain.java:233)
>   at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:60)
>   at org.apache.oozie.action.hadoop.PigMain.main(PigMain.java:73)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:497)
>   at 
> org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:232)
>   at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
>   at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:453)
>   at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
>   at 
> org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.runSubtask(LocalContainerLauncher.java:380)
>   at 
> org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.runTask(LocalContainerLauncher.java:301)
>   at 
> org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.access$200(LocalContainerLauncher.java:187)
>   at 
> org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler$1.run(LocalContainerLauncher.java:230)
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>   at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>   at java.lang.Thread.run(Thread.java:745)
> Caused by: 
>  pig script failed to validate: 
> org.apache.pig.backend.executionengine.ExecException: ERROR 1070: Could not 
> resolve org.apache.hive.hcatalog.pig.HCatLoader using imports: [, java.lang., 
> org.apache.pig.builtin., org.apache.pig.impl.builtin.]
>   at 
> org.apache.pig.parser.LogicalPlanBuilder.validateFuncSpec(LogicalPlanBuilder.java:1339)
>   at 
> org.apache.pig.parser.LogicalPlanBuilder.buildFuncSpec(LogicalPlanBuilder.java:1324)
>   at 
> org.apache.pig.parser.LogicalPlanGenerator.func_clause(LogicalPlanGenerator.java:5184)
>   at 
> org.apache.pig.parser.LogicalPlanGenerator.load_clause(LogicalPlanGenerator.java:3515)
>   at 
> org.apache.pig.parser.LogicalPlanGenerator.op_clause(LogicalPlanGenerator.java:1625)
>   at 
> org.apache.pig.parser.LogicalPlanGenerator.general_statement(LogicalPlanGenerator.java:1102)
>   at 
> org.apache.pig.parser.LogicalPlanGenerator.statement(LogicalPlanGenerator.java:560)
>

[jira] [Commented] (OOZIE-2710) Oozie HCatalog example workflow fails

[Abhishek Bafna (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15594365#comment-15594365
 ] 

Abhishek Bafna commented on OOZIE-2710:
---

Thanks [~shwethags].

> Oozie HCatalog example workflow fails
> -
>
> Key: OOZIE-2710
> URL: https://issues.apache.org/jira/browse/OOZIE-2710
> Project: Oozie
>  Issue Type: Bug
>  Components: core, examples
>Reporter: Abhishek Bafna
>Assignee: Abhishek Bafna
>Priority: Blocker
> Fix For: 4.3.0
>
> Attachments: OOZIE-2710-00.patch
>
>
> Oozie HCatalog example workflow fails with the below exception:
> {noformat}
> 2016-10-12 10:33:37,260 [uber-SubtaskRunner] ERROR org.apache.pig.PigServer  
> - exception during parsing: Error during parsing. Could not resolve 
> org.apache.hive.hcatalog.pig.HCatLoader using imports: [, java.lang., 
> org.apache.pig.builtin., org.apache.pig.impl.builtin.]
> Failed to parse: Pig script failed to parse: 
>  pig script failed to validate: 
> org.apache.pig.backend.executionengine.ExecException: ERROR 1070: Could not 
> resolve org.apache.hive.hcatalog.pig.HCatLoader using imports: [, java.lang., 
> org.apache.pig.builtin., org.apache.pig.impl.builtin.]
>   at 
> org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:199)
>   at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1819)
>   at org.apache.pig.PigServer$Graph.access$000(PigServer.java:1527)
>   at org.apache.pig.PigServer.parseAndBuild(PigServer.java:460)
>   at org.apache.pig.PigServer.executeBatch(PigServer.java:485)
>   at org.apache.pig.PigServer.executeBatch(PigServer.java:471)
>   at 
> org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java:172)
>   at 
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:235)
>   at 
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:206)
>   at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:81)
>   at org.apache.pig.Main.run(Main.java:501)
>   at org.apache.pig.PigRunner.run(PigRunner.java:49)
>   at org.apache.oozie.action.hadoop.PigMain.runPigJob(PigMain.java:290)
>   at org.apache.oozie.action.hadoop.PigMain.run(PigMain.java:233)
>   at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:60)
>   at org.apache.oozie.action.hadoop.PigMain.main(PigMain.java:73)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:497)
>   at 
> org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:232)
>   at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
>   at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:453)
>   at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
>   at 
> org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.runSubtask(LocalContainerLauncher.java:380)
>   at 
> org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.runTask(LocalContainerLauncher.java:301)
>   at 
> org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.access$200(LocalContainerLauncher.java:187)
>   at 
> org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler$1.run(LocalContainerLauncher.java:230)
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>   at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>   at java.lang.Thread.run(Thread.java:745)
> Caused by: 
>  pig script failed to validate: 
> org.apache.pig.backend.executionengine.ExecException: ERROR 1070: Could not 
> resolve org.apache.hive.hcatalog.pig.HCatLoader using imports: [, java.lang., 
> org.apache.pig.builtin., org.apache.pig.impl.builtin.]
>   at 
> org.apache.pig.parser.LogicalPlanBuilder.validateFuncSpec(LogicalPlanBuilder.java:1339)
>   at 
> org.apache.pig.parser.LogicalPlanBuilder.buildFuncSpec(LogicalPlanBuilder.java:1324)
>   at 
> org.apache.pig.parser.LogicalPlanGenerator.func_clause(LogicalPlanGenerator.java:5184)
>   at 
> org.apache.pig.parser.LogicalPlanGenerator.load_clause(LogicalPlanGenerator.java:3515)
>   at 
> org.apache.pig.parser.LogicalPlanGenerator.op_clause(LogicalPlanGenerator.java:1625)
>   at 
> org.apache.pig.parser.LogicalPlanGenerator.general_statement(LogicalPlanGenerator.java:1102)
>   at 
> org.apache.pig.parser.LogicalPla

[jira] [Commented] (OOZIE-2716) Oozie Truncates Workflow Parameter Values which contain '}}'

[Abhishek Bafna (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15594420#comment-15594420
 ] 

Abhishek Bafna commented on OOZIE-2716:
---

I used a simple {{shell}} action which uses this parameter and prints it 
correctly. Though I submitted it from command line.

Did you check the oozie.log for more information, any message it prints 
relevant to your workflow. Thanks.

> Oozie Truncates Workflow Parameter Values which contain '}}'
> 
>
> Key: OOZIE-2716
> URL: https://issues.apache.org/jira/browse/OOZIE-2716
> Project: Oozie
>  Issue Type: Bug
>Affects Versions: 4.1.0
> Environment: Using Oozie 4.1.0-cdh5.8.2, Java 1.8.0_60-b27
>Reporter: Ruben Aguilar
>
> When submitting a workflow job to oozie through the web api, if the value of 
> a substituted workflow parameter contains a {{"\}\}"}} then the rest of the 
> value is truncated. This happens quite often which json data. I tried 
> delimiting by 
> {code:javascript}
> }} => 
> }} => \}\}
> }} => '}''}'
> {code}
> but none of these worked.
> Example:
> Expected value received in {{jar}} (ending double braces):
> {code:javascript}
> {"hdfsArtifacts":null,"azureSqlArtifacts":null,"osdbArtifacts":{"username":"","password":"","server":"","database":"","schema":"","scriptsPath":"","outputType":""}}
> {code}
> Actual value received in {{jar}} (lacking ending double braces):
> {code:javascript}
> {"hdfsArtifacts":null,"azureSqlArtifacts":null,"osdbArtifacts":{"username":"","password":"","server":"","database":"","schema":"","scriptsPath":"","outputType":""
> {code}
> {{workflow.xml}}:
> {code:xml}
> 
>   
>   
> 
>   ${jobTracker}
>   ${nameNode}
>   ${master}
>   cluster
>   ${jobName}
>   ${classToExecute}
>   ${nameNode}${executableJarPath}
>   
>   ${artifactSpecification}
> 
> 
> 
>   
>   
> Spark Action failed, error 
> message[${wf:errorMessage(wf:lastErrorNode())}]
>   
>   
> 
> {code}
> Configuration sent to web api:
> {code:xml}
> 
>   
> jobName
> end-2-end-test
>   
>   
> oozie.use.system.libpath
> True
>   
>   
> executableJarPath
> 
> /oozie-jars/e2e-job-test/version-14/E2EJobTest-assembly-1.20.jar
>   
>   
> user.name
> oozie
>   
>   
> mapreduce.job.user.name
> oozie
>   
>   
> master
> yarn-cluster
>   
>   
> oozie.wf.application.path
> 
> hdfs://some-hdfs:8020/oozie-workflow/e2e-job-test/version-14/workflow.xml
>   
>   
> artifactSpecification
> 
> {"hdfsArtifacts":null,"azureSqlArtifacts":null,"osdbArtifacts":{"username":"","password":"","server":"","database":"","schema":"","scriptsPath":"","outputType":""}}
>   
>   
> jobTracker
> some-namenode:8032
>   
>   
> nameNode
> hdfs://some-namenode:8020
>   
>   
> classToExecute
> E2EJob
>   
> 
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OOZIE-2690) OOZIE NPE while executing kill()

[Abhishek Bafna (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-2690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Abhishek Bafna updated OOZIE-2690:
--
Fix Version/s: 5.0.0

> OOZIE NPE while executing kill()
> 
>
> Key: OOZIE-2690
> URL: https://issues.apache.org/jira/browse/OOZIE-2690
> Project: Oozie
>  Issue Type: Bug
>Reporter: Cheng Xu
>Assignee: Abhishek Bafna
> Fix For: 5.0.0
>
> Attachments: OOZIE-2690-00.patch
>
>
> ION[027-160917110703953-oozie-oozi-W@java2] Exception while executing 
> kill(). Error Code [NullPointerException], Message[NullPointerException: null]
> org.apache.oozie.action.ActionExecutorException: NullPointerException: null
> at 
> org.apache.oozie.action.ActionExecutor.convertException(ActionExecutor.java:446)
> at 
> org.apache.oozie.action.hadoop.JavaActionExecutor.kill(JavaActionExecutor.java:1632)
> at 
> org.apache.oozie.command.wf.ActionKillXCommand.execute(ActionKillXCommand.java:146)
> at 
> org.apache.oozie.command.wf.ActionKillXCommand.execute(ActionKillXCommand.java:62)
> at org.apache.oozie.command.XCommand.call(XCommand.java:287)
> at 
> org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:331)
> at 
> org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:260)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at 
> org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:178)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.NullPointerException
> at org.apache.hadoop.mapreduce.TypeConverter.toYarn(TypeConverter.java:79)
> at 
> org.apache.hadoop.mapred.ClientServiceDelegate.(ClientServiceDelegate.java:120)
> at org.apache.hadoop.mapred.ClientCache.getClient(ClientCache.java:68)
> at org.apache.hadoop.mapred.YARNRunner.getJobStatus(YARNRunner.java:573)
> at org.apache.hadoop.mapreduce.Cluster.getJob(Cluster.java:184)
> at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:601)
> at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:599)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724)
> at 
> org.apache.hadoop.mapred.JobClient.getJobUsingCluster(JobClient.java:599)
> at org.apache.hadoop.mapred.JobClient.getJobInner(JobClient.java:609)
> at org.apache.hadoop.mapred.JobClient.getJob(JobClient.java:639)
> at 
> org.apache.oozie.action.hadoop.JavaActionExecutor.getJobClientWithRetries(JavaActionExecutor.java:1432)
> at 
> org.apache.oozie.action.hadoop.JavaActionExecutor.getRunningJob(JavaActionExecutor.java:1420)
> at 
> org.apache.oozie.action.hadoop.JavaActionExecutor.kill(JavaActionExecutor.java:1623)
> ... 10 more



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OOZIE-2041) Add an admin command to run the PurgeXCommand

[Abhishek Bafna (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-2041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Abhishek Bafna updated OOZIE-2041:
--
Fix Version/s: 5.0.0

> Add an admin command to run the PurgeXCommand
> -
>
> Key: OOZIE-2041
> URL: https://issues.apache.org/jira/browse/OOZIE-2041
> Project: Oozie
>  Issue Type: New Feature
>  Components: core
>Affects Versions: trunk
>Reporter: Robert Kanter
>Assignee: Abhishek Bafna
> Fix For: 5.0.0
>
> Attachments: OOZIE-2041-00.patch, OOZIE-2041-01.patch, 
> OOZIE-2041-02.patch, OOZIE-2041-03.patch
>
>
> Some users may find it useful to be able to run the PurgeXCommand on-demand.  
> We can add a new admin command to run this.  e.g.
> {noformat}
> oozie admin -purge
> {noformat}
> With no args, it can use the "older than" values from oozie-site, but we 
> could have it take some arguments to override those for the command.
> Another thing to worry about is making sure that we don't run this if either 
> it's already running (i.e. the user sent it twice) or if the scheduled 
> PurgeService is already running it.  On top of that, we may need extra 
> consideration for HA setups where we currently only run it on the leader.  
> We should probably also have a to not schedule the PurgeService in ooize-site 
> for users who only want to run it manually.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OOZIE-2674) Improve oozie commads documentation

[Abhishek Bafna (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-2674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Abhishek Bafna updated OOZIE-2674:
--
Fix Version/s: 5.0.0

> Improve oozie commads documentation
> ---
>
> Key: OOZIE-2674
> URL: https://issues.apache.org/jira/browse/OOZIE-2674
> Project: Oozie
>  Issue Type: Bug
>  Components: docs
>Reporter: Abhishek Bafna
>Assignee: Abhishek Bafna
> Fix For: 5.0.0
>
> Attachments: OOZIE-2674-00.patch, OOZIE-2674-01.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OOZIE-2674) Improve oozie commads documentation

[Abhishek Bafna (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-2674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Abhishek Bafna updated OOZIE-2674:
--
Attachment: OOZIE-2674-02.patch

> Improve oozie commads documentation
> ---
>
> Key: OOZIE-2674
> URL: https://issues.apache.org/jira/browse/OOZIE-2674
> Project: Oozie
>  Issue Type: Bug
>  Components: docs
>Reporter: Abhishek Bafna
>Assignee: Abhishek Bafna
> Fix For: 5.0.0
>
> Attachments: OOZIE-2674-00.patch, OOZIE-2674-01.patch, 
> OOZIE-2674-02.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2674) Improve oozie commads documentation

[Abhishek Bafna (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15595925#comment-15595925
 ] 

Abhishek Bafna commented on OOZIE-2674:
---

Thanks [~rkanter] for the feedback and review. I updated the patch.

> Improve oozie commads documentation
> ---
>
> Key: OOZIE-2674
> URL: https://issues.apache.org/jira/browse/OOZIE-2674
> Project: Oozie
>  Issue Type: Bug
>  Components: docs
>Reporter: Abhishek Bafna
>Assignee: Abhishek Bafna
> Fix For: 5.0.0
>
> Attachments: OOZIE-2674-00.patch, OOZIE-2674-01.patch, 
> OOZIE-2674-02.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (OOZIE-2674) Improve oozie commads documentation

[Abhishek Bafna (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15595925#comment-15595925
 ] 

Abhishek Bafna edited comment on OOZIE-2674 at 10/21/16 6:25 PM:
-

Thanks [~rkanter] for the feedback and review. I have updated the patch.


was (Author: abhishekbafna):
Thanks [~rkanter] for the feedback and review. I updated the patch.

> Improve oozie commads documentation
> ---
>
> Key: OOZIE-2674
> URL: https://issues.apache.org/jira/browse/OOZIE-2674
> Project: Oozie
>  Issue Type: Bug
>  Components: docs
>Reporter: Abhishek Bafna
>Assignee: Abhishek Bafna
> Fix For: 5.0.0
>
> Attachments: OOZIE-2674-00.patch, OOZIE-2674-01.patch, 
> OOZIE-2674-02.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2674) Improve oozie commads documentation

[Abhishek Bafna (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15595952#comment-15595952
 ] 

Abhishek Bafna commented on OOZIE-2674:
---

I feel it would be good to take it as usually we not update the site document 
with doc fixes or formatting. :)

> Improve oozie commads documentation
> ---
>
> Key: OOZIE-2674
> URL: https://issues.apache.org/jira/browse/OOZIE-2674
> Project: Oozie
>  Issue Type: Bug
>  Components: docs
>Reporter: Abhishek Bafna
>Assignee: Abhishek Bafna
> Fix For: 5.0.0
>
> Attachments: OOZIE-2674-00.patch, OOZIE-2674-01.patch, 
> OOZIE-2674-02.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2717) oozie starts running workflow even if startdate has already passed

[Abhishek Bafna (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15599212#comment-15599212
 ] 

Abhishek Bafna commented on OOZIE-2717:
---

Oozie coordinators have option for specifying execution order. Default is 
{{FIFO}}, which will executes all the instances in {{FIFO}} manner. If you want 
to skip all the missed or old instances then use {{NONE}} and if you want to 
execute only the last instance from the missed or old instances then use 
{{LAST_ONLY}}. Thanks.

Check Oozie Coordinator documentation for more information : [Synchronous 
Coordinator Application 
Definition|https://oozie.apache.org/docs/4.2.0/CoordinatorFunctionalSpec.html#a6.3._Synchronous_Coordinator_Application_Definition]

> oozie starts running workflow even if startdate has already passed
> --
>
> Key: OOZIE-2717
> URL: https://issues.apache.org/jira/browse/OOZIE-2717
> Project: Oozie
>  Issue Type: Bug
>Reporter: vishal
>
> for ex: if start = today - 30days .and if you run the coordinator today then 
> oozie will start running workflows from past 30 days.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (OOZIE-2681) fix javadoc to compile on JDK8 again

[Abhishek Bafna (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-2681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Abhishek Bafna reassigned OOZIE-2681:
-

Assignee: Abhishek Bafna

> fix javadoc to compile on JDK8 again
> 
>
> Key: OOZIE-2681
> URL: https://issues.apache.org/jira/browse/OOZIE-2681
> Project: Oozie
>  Issue Type: Bug
>  Components: build
>Affects Versions: 4.3.0
>Reporter: Robert Kanter
>Assignee: Abhishek Bafna
>Priority: Critical
> Fix For: 5.0.0
>
>
> We have a lot of Javadoc build failures, which causes Oozie to fail to build 
> on Java 8, which is more strict than Java 6 or 7 was.  OOZIE-2178 fixed all 
> of these that existed at that time, but we've since introduced a lot more.  
> OOZIE-2487 made a workaround to prevent this from causing the build to fail, 
> but we should fix these (again) and remove the OOZIE-2487 workaround.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)