[jira] [Commented] (OOZIE-2582) Populating external child Ids for action failures
[ https://issues.apache.org/jira/browse/OOZIE-2582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15527170#comment-15527170 ] Abhishek Bafna commented on OOZIE-2582: --- Thanks [~rohini] and [~rkanter] for the reviews and committing it. > Populating external child Ids for action failures > - > > Key: OOZIE-2582 > URL: https://issues.apache.org/jira/browse/OOZIE-2582 > Project: Oozie > Issue Type: Bug > Components: core >Reporter: Abhishek Bafna >Assignee: Abhishek Bafna > Fix For: 4.3.0 > > Attachments: OOZIE-2582-00.patch, OOZIE-2582-01.patch, > OOZIE-2582-02.patch, OOZIE-2582-03.patch, OOZIE-2582-04.patch, > OOZIE-2582-05.patch, OOZIE-2582-06.patch, OOZIE-2582-07.patch, > OOZIE-2582-08.patch, OOZIE-2582-09.patch > > > Currently Oozie external child ids are populated into workflow bean, when the > job/action completes successfully. It should populate external child ids in > case of job failures as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OOZIE-2582) Populating external child Ids for action failures
[
https://issues.apache.org/jira/browse/OOZIE-2582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15519321#comment-15519321
]
Abhishek Bafna commented on OOZIE-2582:
---
It should work for failure cases, as it is calling the
{{writeExternalChildIDs}} in the {{writeExternalChildIDs}} block. I have added
the {{Pattern}} for Hadoop 2 as well and added comment for external child id
check, currently they are disabled. Thanks.
> Populating external child Ids for action failures
> -
>
> Key: OOZIE-2582
> URL: https://issues.apache.org/jira/browse/OOZIE-2582
> Project: Oozie
> Issue Type: Bug
> Components: core
>Reporter: Abhishek Bafna
>Assignee: Abhishek Bafna
> Fix For: 4.3.0
>
> Attachments: OOZIE-2582-00.patch, OOZIE-2582-01.patch,
> OOZIE-2582-02.patch, OOZIE-2582-03.patch, OOZIE-2582-04.patch,
> OOZIE-2582-05.patch, OOZIE-2582-06.patch, OOZIE-2582-07.patch,
> OOZIE-2582-08.patch, OOZIE-2582-09.patch
>
>
> Currently Oozie external child ids are populated into workflow bean, when the
> job/action completes successfully. It should populate external child ids in
> case of job failures as well.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OOZIE-2582) Populating external child Ids for action failures
[
https://issues.apache.org/jira/browse/OOZIE-2582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15518682#comment-15518682
]
Hadoop QA commented on OOZIE-2582:
--
Testing JIRA OOZIE-2582
Cleaning local git workspace
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:green}+1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than
132
.{color:green}+1{color} the patch does adds/modifies 5 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
warnings
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac
warnings
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:green}+1 TESTS{color}
.Tests run: 1808
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch
{color:green}*+1 Overall result, good!, no -1s*{color}
The full output of the test-patch run is available at
. https://builds.apache.org/job/oozie-trunk-precommit-build/3329/
> Populating external child Ids for action failures
> -
>
> Key: OOZIE-2582
> URL: https://issues.apache.org/jira/browse/OOZIE-2582
> Project: Oozie
> Issue Type: Bug
> Components: core
>Reporter: Abhishek Bafna
>Assignee: Abhishek Bafna
> Fix For: 4.3.0
>
> Attachments: OOZIE-2582-00.patch, OOZIE-2582-01.patch,
> OOZIE-2582-02.patch, OOZIE-2582-03.patch, OOZIE-2582-04.patch,
> OOZIE-2582-05.patch, OOZIE-2582-06.patch, OOZIE-2582-07.patch,
> OOZIE-2582-08.patch, OOZIE-2582-09.patch
>
>
> Currently Oozie external child ids are populated into workflow bean, when the
> job/action completes successfully. It should populate external child ids in
> case of job failures as well.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OOZIE-2582) Populating external child Ids for action failures
[
https://issues.apache.org/jira/browse/OOZIE-2582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15517968#comment-15517968
]
Hadoop QA commented on OOZIE-2582:
--
Testing JIRA OOZIE-2582
Cleaning local git workspace
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:green}+1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than
132
.{color:green}+1{color} the patch does adds/modifies 5 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
warnings
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac
warnings
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:green}+1 TESTS{color}
.Tests run: 1808
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch
{color:green}*+1 Overall result, good!, no -1s*{color}
The full output of the test-patch run is available at
. https://builds.apache.org/job/oozie-trunk-precommit-build/3325/
> Populating external child Ids for action failures
> -
>
> Key: OOZIE-2582
> URL: https://issues.apache.org/jira/browse/OOZIE-2582
> Project: Oozie
> Issue Type: Bug
> Components: core
>Reporter: Abhishek Bafna
>Assignee: Abhishek Bafna
> Fix For: 4.3.0
>
> Attachments: OOZIE-2582-00.patch, OOZIE-2582-01.patch,
> OOZIE-2582-02.patch, OOZIE-2582-03.patch, OOZIE-2582-04.patch,
> OOZIE-2582-05.patch, OOZIE-2582-06.patch, OOZIE-2582-07.patch,
> OOZIE-2582-08.patch
>
>
> Currently Oozie external child ids are populated into workflow bean, when the
> job/action completes successfully. It should populate external child ids in
> case of job failures as well.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OOZIE-2582) Populating external child Ids for action failures
[ https://issues.apache.org/jira/browse/OOZIE-2582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15517562#comment-15517562 ] Robert Kanter commented on OOZIE-2582: -- I'll try to take another look at OOZIE-2613 soon. For this patch, I think we should try to update Hive2Main as well so that it can get external child Ids when there's a failure there. > Populating external child Ids for action failures > - > > Key: OOZIE-2582 > URL: https://issues.apache.org/jira/browse/OOZIE-2582 > Project: Oozie > Issue Type: Bug > Components: core >Reporter: Abhishek Bafna >Assignee: Abhishek Bafna > Fix For: 4.3.0 > > Attachments: OOZIE-2582-00.patch, OOZIE-2582-01.patch, > OOZIE-2582-02.patch, OOZIE-2582-03.patch, OOZIE-2582-04.patch, > OOZIE-2582-05.patch, OOZIE-2582-06.patch, OOZIE-2582-07.patch, > OOZIE-2582-08.patch > > > Currently Oozie external child ids are populated into workflow bean, when the > job/action completes successfully. It should populate external child ids in > case of job failures as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OOZIE-2582) Populating external child Ids for action failures
[ https://issues.apache.org/jira/browse/OOZIE-2582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15517256#comment-15517256 ] Abhishek Bafna commented on OOZIE-2582: --- [~rkanter] Does this means we will bump the priority for OOZIE-2613. Thanks. :) > Populating external child Ids for action failures > - > > Key: OOZIE-2582 > URL: https://issues.apache.org/jira/browse/OOZIE-2582 > Project: Oozie > Issue Type: Bug > Components: core >Reporter: Abhishek Bafna >Assignee: Abhishek Bafna > Fix For: 4.3.0 > > Attachments: OOZIE-2582-00.patch, OOZIE-2582-01.patch, > OOZIE-2582-02.patch, OOZIE-2582-03.patch, OOZIE-2582-04.patch, > OOZIE-2582-05.patch, OOZIE-2582-06.patch, OOZIE-2582-07.patch, > OOZIE-2582-08.patch > > > Currently Oozie external child ids are populated into workflow bean, when the > job/action completes successfully. It should populate external child ids in > case of job failures as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OOZIE-2582) Populating external child Ids for action failures
[ https://issues.apache.org/jira/browse/OOZIE-2582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15517249#comment-15517249 ] Robert Kanter commented on OOZIE-2582: -- OOZIE-1964 added support for the Hive2 Action to get child Job IDs from Beeline, if you're using a version of Hive that has HIVE-7615. Oozie is currently set to use Hive 0.13.1 and HIVE-7615 is in 0.14.0, so that's probably why you're not seeing it. > Populating external child Ids for action failures > - > > Key: OOZIE-2582 > URL: https://issues.apache.org/jira/browse/OOZIE-2582 > Project: Oozie > Issue Type: Bug > Components: core >Reporter: Abhishek Bafna >Assignee: Abhishek Bafna > Fix For: 4.3.0 > > Attachments: OOZIE-2582-00.patch, OOZIE-2582-01.patch, > OOZIE-2582-02.patch, OOZIE-2582-03.patch, OOZIE-2582-04.patch, > OOZIE-2582-05.patch, OOZIE-2582-06.patch, OOZIE-2582-07.patch, > OOZIE-2582-08.patch > > > Currently Oozie external child ids are populated into workflow bean, when the > job/action completes successfully. It should populate external child ids in > case of job failures as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OOZIE-2582) Populating external child Ids for action failures
[ https://issues.apache.org/jira/browse/OOZIE-2582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15517232#comment-15517232 ] Rohini Palaniswamy commented on OOZIE-2582: --- [~rkanter] is the authority on this one. Have never used hive2 action. > Populating external child Ids for action failures > - > > Key: OOZIE-2582 > URL: https://issues.apache.org/jira/browse/OOZIE-2582 > Project: Oozie > Issue Type: Bug > Components: core >Reporter: Abhishek Bafna >Assignee: Abhishek Bafna > Fix For: 4.3.0 > > Attachments: OOZIE-2582-00.patch, OOZIE-2582-01.patch, > OOZIE-2582-02.patch, OOZIE-2582-03.patch, OOZIE-2582-04.patch, > OOZIE-2582-05.patch, OOZIE-2582-06.patch, OOZIE-2582-07.patch, > OOZIE-2582-08.patch > > > Currently Oozie external child ids are populated into workflow bean, when the > job/action completes successfully. It should populate external child ids in > case of job failures as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OOZIE-2582) Populating external child Ids for action failures
[ https://issues.apache.org/jira/browse/OOZIE-2582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15517104#comment-15517104 ] Abhishek Bafna commented on OOZIE-2582: --- [~rohini], [~rkanter] The external child id population does not work for Hive2 action. Hive2 uses the beeline client and submits the query to HiveServer2 for execution and beeline does not get any information about the launched jobs. Please correct me, If my understanding is wrong. Thanks. > Populating external child Ids for action failures > - > > Key: OOZIE-2582 > URL: https://issues.apache.org/jira/browse/OOZIE-2582 > Project: Oozie > Issue Type: Bug > Components: core >Reporter: Abhishek Bafna >Assignee: Abhishek Bafna > Fix For: 4.3.0 > > Attachments: OOZIE-2582-00.patch, OOZIE-2582-01.patch, > OOZIE-2582-02.patch, OOZIE-2582-03.patch, OOZIE-2582-04.patch, > OOZIE-2582-05.patch, OOZIE-2582-06.patch, OOZIE-2582-07.patch, > OOZIE-2582-08.patch > > > Currently Oozie external child ids are populated into workflow bean, when the > job/action completes successfully. It should populate external child ids in > case of job failures as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OOZIE-2582) Populating external child Ids for action failures
[
https://issues.apache.org/jira/browse/OOZIE-2582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15517093#comment-15517093
]
Abhishek Bafna commented on OOZIE-2582:
---
The Hive2 testcase failed with {{default}} database not found. I guess I need
to move the new negative testcase into the existing {{testHive2Action}} similar
to one which it already have.
{{TestCoordActionsKillXCommand}} testcases passes locally.
> Populating external child Ids for action failures
> -
>
> Key: OOZIE-2582
> URL: https://issues.apache.org/jira/browse/OOZIE-2582
> Project: Oozie
> Issue Type: Bug
> Components: core
>Reporter: Abhishek Bafna
>Assignee: Abhishek Bafna
> Fix For: 4.3.0
>
> Attachments: OOZIE-2582-00.patch, OOZIE-2582-01.patch,
> OOZIE-2582-02.patch, OOZIE-2582-03.patch, OOZIE-2582-04.patch,
> OOZIE-2582-05.patch, OOZIE-2582-06.patch, OOZIE-2582-07.patch
>
>
> Currently Oozie external child ids are populated into workflow bean, when the
> job/action completes successfully. It should populate external child ids in
> case of job failures as well.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OOZIE-2582) Populating external child Ids for action failures
[
https://issues.apache.org/jira/browse/OOZIE-2582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15516969#comment-15516969
]
Hadoop QA commented on OOZIE-2582:
--
Testing JIRA OOZIE-2582
Cleaning local git workspace
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:green}+1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than
132
.{color:green}+1{color} the patch does adds/modifies 5 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
warnings
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac
warnings
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:red}-1 TESTS{color}
.Tests run: 1809
.Tests failed: 2
.Tests errors: 0
.The patch failed the following testcases:
.
testActionKillCommandActionNumbers(org.apache.oozie.command.coord.TestCoordActionsKillXCommand)
. testHive2Action(org.apache.oozie.action.hadoop.TestHive2ActionExecutor)
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch
{color:red}*-1 Overall result, please check the reported -1(s)*{color}
The full output of the test-patch run is available at
. https://builds.apache.org/job/oozie-trunk-precommit-build/3324/
> Populating external child Ids for action failures
> -
>
> Key: OOZIE-2582
> URL: https://issues.apache.org/jira/browse/OOZIE-2582
> Project: Oozie
> Issue Type: Bug
> Components: core
>Reporter: Abhishek Bafna
>Assignee: Abhishek Bafna
> Fix For: 4.3.0
>
> Attachments: OOZIE-2582-00.patch, OOZIE-2582-01.patch,
> OOZIE-2582-02.patch, OOZIE-2582-03.patch, OOZIE-2582-04.patch,
> OOZIE-2582-05.patch, OOZIE-2582-06.patch, OOZIE-2582-07.patch
>
>
> Currently Oozie external child ids are populated into workflow bean, when the
> job/action completes successfully. It should populate external child ids in
> case of job failures as well.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OOZIE-2582) Populating external child Ids for action failures
[
https://issues.apache.org/jira/browse/OOZIE-2582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15514238#comment-15514238
]
Rohini Palaniswamy commented on OOZIE-2582:
---
Thanks for the cleanup. Have few more comments.
- Remove comment // See JavaActionExecutor#readExternalChildIDs for how they
are read
- getHadoopJobIds should just return String and if it is null then
writeExternalChildIds should skip creating the file
- You are now creating string as "job_id1, job_id2" instead of
"job_id1,job_id2". i.e extra space after comma which will require trimming
later. Instead of jobIds.toString().substring(1,
jobIds.toString().length()-1), please do
{code}
if (jobIds.isEmpty()) {
return null;
}
else {
return StringUtils.join(",");
}
{code}
> Populating external child Ids for action failures
> -
>
> Key: OOZIE-2582
> URL: https://issues.apache.org/jira/browse/OOZIE-2582
> Project: Oozie
> Issue Type: Bug
> Components: core
>Reporter: Abhishek Bafna
>Assignee: Abhishek Bafna
> Fix For: 4.3.0
>
> Attachments: OOZIE-2582-00.patch, OOZIE-2582-01.patch,
> OOZIE-2582-02.patch, OOZIE-2582-03.patch, OOZIE-2582-04.patch,
> OOZIE-2582-05.patch, OOZIE-2582-06.patch
>
>
> Currently Oozie external child ids are populated into workflow bean, when the
> job/action completes successfully. It should populate external child ids in
> case of job failures as well.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OOZIE-2582) Populating external child Ids for action failures
[ https://issues.apache.org/jira/browse/OOZIE-2582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15513759#comment-15513759 ] Abhishek Bafna commented on OOZIE-2582: --- Ping: [~rohini], [~rkanter]. Pinging for Oozie 4.3.0 Release perspective. > Populating external child Ids for action failures > - > > Key: OOZIE-2582 > URL: https://issues.apache.org/jira/browse/OOZIE-2582 > Project: Oozie > Issue Type: Bug > Components: core >Reporter: Abhishek Bafna >Assignee: Abhishek Bafna > Fix For: 4.3.0 > > Attachments: OOZIE-2582-00.patch, OOZIE-2582-01.patch, > OOZIE-2582-02.patch, OOZIE-2582-03.patch, OOZIE-2582-04.patch, > OOZIE-2582-05.patch, OOZIE-2582-06.patch > > > Currently Oozie external child ids are populated into workflow bean, when the > job/action completes successfully. It should populate external child ids in > case of job failures as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OOZIE-2582) Populating external child Ids for action failures
[ https://issues.apache.org/jira/browse/OOZIE-2582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15507523#comment-15507523 ] Abhishek Bafna commented on OOZIE-2582: --- Thanks for the feedback [~rohini]. Attached the updated patch here and RB as well. > Populating external child Ids for action failures > - > > Key: OOZIE-2582 > URL: https://issues.apache.org/jira/browse/OOZIE-2582 > Project: Oozie > Issue Type: Bug > Components: core >Reporter: Abhishek Bafna >Assignee: Abhishek Bafna > Fix For: 4.3.0 > > Attachments: OOZIE-2582-00.patch, OOZIE-2582-01.patch, > OOZIE-2582-02.patch, OOZIE-2582-03.patch, OOZIE-2582-04.patch, > OOZIE-2582-05.patch, OOZIE-2582-06.patch > > > Currently Oozie external child ids are populated into workflow bean, when the > job/action completes successfully. It should populate external child ids in > case of job failures as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OOZIE-2582) Populating external child Ids for action failures
[ https://issues.apache.org/jira/browse/OOZIE-2582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15506915#comment-15506915 ] Abhishek Bafna commented on OOZIE-2582: --- Testcase passes locally. > Populating external child Ids for action failures > - > > Key: OOZIE-2582 > URL: https://issues.apache.org/jira/browse/OOZIE-2582 > Project: Oozie > Issue Type: Bug > Components: core >Reporter: Abhishek Bafna >Assignee: Abhishek Bafna > Fix For: 4.3.0 > > Attachments: OOZIE-2582-00.patch, OOZIE-2582-01.patch, > OOZIE-2582-02.patch, OOZIE-2582-03.patch, OOZIE-2582-04.patch, > OOZIE-2582-05.patch, OOZIE-2582-06.patch > > > Currently Oozie external child ids are populated into workflow bean, when the > job/action completes successfully. It should populate external child ids in > case of job failures as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OOZIE-2582) Populating external child Ids for action failures
[
https://issues.apache.org/jira/browse/OOZIE-2582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15506844#comment-15506844
]
Hadoop QA commented on OOZIE-2582:
--
Testing JIRA OOZIE-2582
Cleaning local git workspace
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:green}+1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than
132
.{color:green}+1{color} the patch does adds/modifies 4 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
warnings
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac
warnings
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:red}-1 TESTS{color}
.Tests run: 1808
.Tests failed: 1
.Tests errors: 0
.The patch failed the following testcases:
. testRecovery(org.apache.oozie.action.hadoop.TestJavaActionExecutor)
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch
{color:red}*-1 Overall result, please check the reported -1(s)*{color}
The full output of the test-patch run is available at
. https://builds.apache.org/job/oozie-trunk-precommit-build/3302/
> Populating external child Ids for action failures
> -
>
> Key: OOZIE-2582
> URL: https://issues.apache.org/jira/browse/OOZIE-2582
> Project: Oozie
> Issue Type: Bug
> Components: core
>Reporter: Abhishek Bafna
>Assignee: Abhishek Bafna
> Fix For: 4.3.0
>
> Attachments: OOZIE-2582-00.patch, OOZIE-2582-01.patch,
> OOZIE-2582-02.patch, OOZIE-2582-03.patch, OOZIE-2582-04.patch,
> OOZIE-2582-05.patch, OOZIE-2582-06.patch
>
>
> Currently Oozie external child ids are populated into workflow bean, when the
> job/action completes successfully. It should populate external child ids in
> case of job failures as well.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OOZIE-2582) Populating external child Ids for action failures
[
https://issues.apache.org/jira/browse/OOZIE-2582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15504558#comment-15504558
]
Rohini Palaniswamy commented on OOZIE-2582:
---
Few comments:
1) Can you move below declarations from PigMain to LauncherMain and replace
references of oozie.action.externalChildIDs with that constant?
{code}
public static final String ACTION_PREFIX = "oozie.action.";
public static final String EXTERNAL_CHILD_IDS = ACTION_PREFIX +
"externalChildIDs";
public static final String EXTERNAL_ACTION_STATS = ACTION_PREFIX +
"stats.properties";
public static final String EXTERNAL_STATS_WRITE = ACTION_PREFIX +
"external.stats.write";
{code}
2)
https://issues.apache.org/jira/browse/OOZIE-2503?focusedCommentId=15315945=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15315945
Distcp and Spark child job url was added by [~satishsaley] only in this release
(OOZIE-2471 and OOZIE-2503) in Oozie 4.3. But they were implemented by
overwriting getCaptureOutput to true and implementing getActionData and writing
to output.properties instead of externalChildIDs. Had asked him to create
OOZIE-2561 to remove that and write directly to externalChildIDs. Can you take
care of that in this patch? HiveMain, Hive2Main and SqoopMain also need to be
changed similarly but the code existed in previous releases. So for jobs that
were launched before upgrade, we need to still read from output.properties. i.e
{code}
String externalIDs =
actionData.get(LauncherMapper.ACTION_DATA_EXTERNAL_CHILD_IDS);
if (externalIDs != null) {
context.setExternalChildIDs(externalIDs);
LOG.info(XLog.STD, "Hadoop Jobs launched : [{0}]",
externalIDs);
}
else if (LauncherMapperHelper.hasOutputData(actionData)) {
// Load stored Hadoop jobs ids and promote them as external child
ids
// This is for jobs launched with older release during upgrade to
Oozie 4.3
Properties props = PropertiesUtils.stringToProperties(actionData
.get(LauncherMapper.ACTION_DATA_OUTPUT_PROPS)
if (props.get(LauncherMain.HADOOP_JOBS) != null) {
context.setExternalChildIDs((String)
props.get(LauncherMain.HADOOP_JOBS));
LOG.info(XLog.STD, "Hadoop Jobs launched : [{0}]", externalIDs);
}
}
{code}
Your current patch writes to externalChildIds in addition to the properties
file. With above change you need to only write to externalChildIds file. Please
remove readExternalChildIDs in JavaActionExecutor as well as it will be unused
after above change. This will also avoid duplication and wastage of storing
child ids in two fields (action output and external child ids) of
WorkflowAction in the database.
4) Can you put the job ids in a LinkedHashSet (to maintain the order in which
jobs were launched) first and then create the string? Doing toString() and
contains check every time is inefficient.
{code}
if (!sb.toString().contains(jobId)) {
+sb.append(separator).append(jobId);
+}
{code}
> Populating external child Ids for action failures
> -
>
> Key: OOZIE-2582
> URL: https://issues.apache.org/jira/browse/OOZIE-2582
> Project: Oozie
> Issue Type: Bug
> Components: core
>Reporter: Abhishek Bafna
>Assignee: Abhishek Bafna
> Fix For: 4.3.0
>
> Attachments: OOZIE-2582-00.patch, OOZIE-2582-01.patch,
> OOZIE-2582-02.patch, OOZIE-2582-03.patch, OOZIE-2582-04.patch,
> OOZIE-2582-05.patch
>
>
> Currently Oozie external child ids are populated into workflow bean, when the
> job/action completes successfully. It should populate external child ids in
> case of job failures as well.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OOZIE-2582) Populating external child Ids for action failures
[ https://issues.apache.org/jira/browse/OOZIE-2582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15494154#comment-15494154 ] Abhishek Bafna commented on OOZIE-2582: --- Ping: [~rkanter], [~jaydeepvishwakarma] > Populating external child Ids for action failures > - > > Key: OOZIE-2582 > URL: https://issues.apache.org/jira/browse/OOZIE-2582 > Project: Oozie > Issue Type: Bug > Components: core >Reporter: Abhishek Bafna >Assignee: Abhishek Bafna > Fix For: 4.3.0 > > Attachments: OOZIE-2582-00.patch, OOZIE-2582-01.patch, > OOZIE-2582-02.patch, OOZIE-2582-03.patch, OOZIE-2582-04.patch, > OOZIE-2582-05.patch > > > Currently Oozie external child ids are populated into workflow bean, when the > job/action completes successfully. It should populate external child ids in > case of job failures as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OOZIE-2582) Populating external child Ids for action failures
[ https://issues.apache.org/jira/browse/OOZIE-2582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15399123#comment-15399123 ] abhishek bafna commented on OOZIE-2582: --- [~satishsaley] Thanks for the initial feedback. If there is anything please provide your feedback. [~puru], [~rkanter] Can you please take a look. Thanks. > Populating external child Ids for action failures > - > > Key: OOZIE-2582 > URL: https://issues.apache.org/jira/browse/OOZIE-2582 > Project: Oozie > Issue Type: Bug > Components: core >Reporter: abhishek bafna >Assignee: abhishek bafna > Attachments: OOZIE-2582-00.patch, OOZIE-2582-01.patch, > OOZIE-2582-02.patch, OOZIE-2582-03.patch, OOZIE-2582-04.patch, > OOZIE-2582-05.patch > > > Currently Oozie external child ids are populated into workflow bean, when the > job/action completes successfully. It should populate external child ids in > case of job failures as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OOZIE-2582) Populating external child Ids for action failures
[
https://issues.apache.org/jira/browse/OOZIE-2582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15397312#comment-15397312
]
Hadoop QA commented on OOZIE-2582:
--
Testing JIRA OOZIE-2582
Cleaning local git workspace
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:green}+1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than
132
.{color:green}+1{color} the patch does adds/modifies 2 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
warnings
{color:red}-1 COMPILE{color}
.{color:red}-1{color} HEAD does not compile
.{color:red}-1{color} patch does not compile
.{color:green}+1{color} the patch does not seem to introduce new javac
warnings
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:red}-1 TESTS{color}
.Tests run: 1792
.Tests failed: 1
.Tests errors: 0
.The patch failed the following testcases:
. testHive2Action(org.apache.oozie.action.hadoop.TestHive2ActionExecutor)
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch
{color:red}*-1 Overall result, please check the reported -1(s)*{color}
The full output of the test-patch run is available at
. https://builds.apache.org/job/oozie-trunk-precommit-build/3161/
> Populating external child Ids for action failures
> -
>
> Key: OOZIE-2582
> URL: https://issues.apache.org/jira/browse/OOZIE-2582
> Project: Oozie
> Issue Type: Bug
> Components: core
>Reporter: abhishek bafna
>Assignee: abhishek bafna
> Attachments: OOZIE-2582-00.patch, OOZIE-2582-01.patch,
> OOZIE-2582-02.patch, OOZIE-2582-03.patch, OOZIE-2582-04.patch,
> OOZIE-2582-05.patch
>
>
> Currently Oozie external child ids are populated into workflow bean, when the
> job/action completes successfully. It should populate external child ids in
> case of job failures as well.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OOZIE-2582) Populating external child Ids for action failures
[
https://issues.apache.org/jira/browse/OOZIE-2582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15396818#comment-15396818
]
Hadoop QA commented on OOZIE-2582:
--
Testing JIRA OOZIE-2582
Cleaning local git workspace
{color:red}-1{color} Patch failed to apply to head of branch
> Populating external child Ids for action failures
> -
>
> Key: OOZIE-2582
> URL: https://issues.apache.org/jira/browse/OOZIE-2582
> Project: Oozie
> Issue Type: Bug
> Components: core
>Reporter: abhishek bafna
>Assignee: abhishek bafna
> Attachments: OOZIE-2582-00.patch, OOZIE-2582-01.patch,
> OOZIE-2582-02.patch, OOZIE-2582-03.patch, OOZIE-2582-04.patch
>
>
> Currently Oozie external child ids are populated into workflow bean, when the
> job/action completes successfully. It should populate external child ids in
> case of job failures as well.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OOZIE-2582) Populating external child Ids for action failures
[ https://issues.apache.org/jira/browse/OOZIE-2582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15370838#comment-15370838 ] abhishek bafna commented on OOZIE-2582: --- Testcase failure is not related to the patch. > Populating external child Ids for action failures > - > > Key: OOZIE-2582 > URL: https://issues.apache.org/jira/browse/OOZIE-2582 > Project: Oozie > Issue Type: Bug > Components: core >Reporter: abhishek bafna >Assignee: abhishek bafna > Attachments: OOZIE-2582-00.patch, OOZIE-2582-01.patch, > OOZIE-2582-02.patch, OOZIE-2582-03.patch, OOZIE-2582-04.patch > > > Currently Oozie external child ids are populated into workflow bean, when the > job/action completes successfully. It should populate external child ids in > case of job failures as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OOZIE-2582) Populating external child Ids for action failures
[
https://issues.apache.org/jira/browse/OOZIE-2582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15369847#comment-15369847
]
Hadoop QA commented on OOZIE-2582:
--
Testing JIRA OOZIE-2582
Cleaning local git workspace
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:green}+1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than
132
.{color:green}+1{color} the patch does adds/modifies 2 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
warnings
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac
warnings
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:red}-1 TESTS{color}
.Tests run: 1788
.Tests failed: 1
.Tests errors: 0
.The patch failed the following testcases:
. testHive2Action(org.apache.oozie.action.hadoop.TestHive2ActionExecutor)
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch
{color:red}*-1 Overall result, please check the reported -1(s)*{color}
The full output of the test-patch run is available at
. https://builds.apache.org/job/oozie-trunk-precommit-build/3033/
> Populating external child Ids for action failures
> -
>
> Key: OOZIE-2582
> URL: https://issues.apache.org/jira/browse/OOZIE-2582
> Project: Oozie
> Issue Type: Bug
> Components: core
>Reporter: abhishek bafna
>Assignee: abhishek bafna
> Attachments: OOZIE-2582-00.patch, OOZIE-2582-01.patch,
> OOZIE-2582-02.patch, OOZIE-2582-03.patch, OOZIE-2582-04.patch
>
>
> Currently Oozie external child ids are populated into workflow bean, when the
> job/action completes successfully. It should populate external child ids in
> case of job failures as well.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OOZIE-2582) Populating external child Ids for action failures
[
https://issues.apache.org/jira/browse/OOZIE-2582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15367585#comment-15367585
]
Hadoop QA commented on OOZIE-2582:
--
Testing JIRA OOZIE-2582
Cleaning local git workspace
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:green}+1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than
132
.{color:green}+1{color} the patch does adds/modifies 2 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
warnings
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac
warnings
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:red}-1 TESTS{color}
.Tests run: 1788
.Tests failed: 3
.Tests errors: 0
.The patch failed the following testcases:
. testDistCpFile(org.apache.oozie.action.hadoop.TestDistCpActionExecutor)
. testSqoopAction(org.apache.oozie.action.hadoop.TestSqoopActionExecutor)
.
testSqoopActionFreeFormQuery(org.apache.oozie.action.hadoop.TestSqoopActionExecutor)
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch
{color:red}*-1 Overall result, please check the reported -1(s)*{color}
The full output of the test-patch run is available at
. https://builds.apache.org/job/oozie-trunk-precommit-build/3030/
> Populating external child Ids for action failures
> -
>
> Key: OOZIE-2582
> URL: https://issues.apache.org/jira/browse/OOZIE-2582
> Project: Oozie
> Issue Type: Bug
> Components: core
>Reporter: abhishek bafna
>Assignee: abhishek bafna
> Attachments: OOZIE-2582-00.patch, OOZIE-2582-01.patch,
> OOZIE-2582-02.patch, OOZIE-2582-03.patch
>
>
> Currently Oozie external child ids are populated into workflow bean, when the
> job/action completes successfully. It should populate external child ids in
> case of job failures as well.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OOZIE-2582) Populating external child Ids for action failures
[ https://issues.apache.org/jira/browse/OOZIE-2582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15366018#comment-15366018 ] abhishek bafna commented on OOZIE-2582: --- The testcase failure is not related. > Populating external child Ids for action failures > - > > Key: OOZIE-2582 > URL: https://issues.apache.org/jira/browse/OOZIE-2582 > Project: Oozie > Issue Type: Bug > Components: core >Reporter: abhishek bafna >Assignee: abhishek bafna > Attachments: OOZIE-2582-00.patch, OOZIE-2582-01.patch, > OOZIE-2582-02.patch > > > Currently Oozie external child ids are populated into workflow bean, when the > job/action completes successfully. It should populate external child ids in > case of job failures as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OOZIE-2582) Populating external child Ids for action failures
[ https://issues.apache.org/jira/browse/OOZIE-2582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15364774#comment-15364774 ] abhishek bafna commented on OOZIE-2582: --- Hi [~satishsaley], I was looking into the Oozie bugs and found that, there exists one Jira for the same issue, reported by you: OOZIE-2561. I was not aware of that Jira. I have uploaded a patch here. I hope you would like to take a look. Thanks Abhishek. > Populating external child Ids for action failures > - > > Key: OOZIE-2582 > URL: https://issues.apache.org/jira/browse/OOZIE-2582 > Project: Oozie > Issue Type: Bug > Components: core >Reporter: abhishek bafna >Assignee: abhishek bafna > Attachments: OOZIE-2582-00.patch, OOZIE-2582-01.patch > > > Currently Oozie external child ids are populated into workflow bean, when the > job/action completes successfully. It should populate external child ids in > case of job failures as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OOZIE-2582) Populating external child Ids for action failures
[
https://issues.apache.org/jira/browse/OOZIE-2582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15343988#comment-15343988
]
abhishek bafna commented on OOZIE-2582:
---
We call {{LauncherMain#writeExternalChildIDs()}} in the finally block for Oozie
actions Main class. Previously, we were writing the data (launched job ids) to
output properties files only. When a launched action job completes
successfully, the external child id is set from
{{JavaActionExecutor#getActionData()}} as part of
{{JavaActionExecutor#check()}}.
The patch contains the changes to write the returned hadoop ids
{{LauncherMain#getHadoopJobIds()}} into the
{{LauncherMapper#ACTION_DATA_EXTERNAL_CHILD_IDS}} file. Because
{{JavaActionExecutor#check}} reads the property and set the external child for
completed actions. This approach should help to capture the external child id
for all the actions.
Also, contains the log filter changes for (Hive and Sqoop), to get the
application id as soon as it is submitted.
> Populating external child Ids for action failures
> -
>
> Key: OOZIE-2582
> URL: https://issues.apache.org/jira/browse/OOZIE-2582
> Project: Oozie
> Issue Type: Bug
> Components: core
>Reporter: abhishek bafna
>Assignee: abhishek bafna
> Attachments: OOZIE-2582-00.patch
>
>
> Currently Oozie external child ids are populated into workflow bean, when the
> job/action completes successfully. It should populate external child ids in
> case of job failures as well.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OOZIE-2582) Populating external child Ids for action failures
[
https://issues.apache.org/jira/browse/OOZIE-2582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15341923#comment-15341923
]
Hadoop QA commented on OOZIE-2582:
--
Testing JIRA OOZIE-2582
Cleaning local git workspace
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:green}+1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than
132
.{color:green}+1{color} the patch does adds/modifies 1 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
warnings
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac
warnings
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:green}+1 TESTS{color}
.Tests run: 1787
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch
{color:green}*+1 Overall result, good!, no -1s*{color}
The full output of the test-patch run is available at
. https://builds.apache.org/job/oozie-trunk-precommit-build/3010/
> Populating external child Ids for action failures
> -
>
> Key: OOZIE-2582
> URL: https://issues.apache.org/jira/browse/OOZIE-2582
> Project: Oozie
> Issue Type: Bug
> Components: core
>Reporter: abhishek bafna
>Assignee: abhishek bafna
> Attachments: OOZIE-2582-00.patch
>
>
> Currently Oozie external child ids are populated into workflow bean, when the
> job/action completes successfully. It should populate external child ids in
> case of job failures as well.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
