[jira] [Updated] (OOZIE-3690) [server] Fix current SpotBugs discovered issues in Oozie's server module
[ https://issues.apache.org/jira/browse/OOZIE-3690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Janos Makai updated OOZIE-3690: --- Attachment: OOZIE-3690-002.patch > [server] Fix current SpotBugs discovered issues in Oozie's server module > > > Key: OOZIE-3690 > URL: https://issues.apache.org/jira/browse/OOZIE-3690 > Project: Oozie > Issue Type: Sub-task > Components: build >Affects Versions: 5.2.1 >Reporter: Janos Makai >Assignee: Janos Makai >Priority: Major > Attachments: OOZIE-3690-001.patch, OOZIE-3690-002.patch, > Oozie-server-SpotBugs-issues.txt > > > This ticket is created for fixing or filtering the SpotBugs related issues in > Oozie's server module. > The exact SpotBugs output is attached for this module. > There are currently 10 SpotBugs discovered issues in this module. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (OOZIE-3698) [sharelib-hcatalog] Fix current SpotBugs discovered issues in Oozie's sharelib-hcatalog module
[ https://issues.apache.org/jira/browse/OOZIE-3698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Janos Makai updated OOZIE-3698: --- Attachment: Oozie-oozie-sharelib-hcatalog-SpotBugs-issues.txt > [sharelib-hcatalog] Fix current SpotBugs discovered issues in Oozie's > sharelib-hcatalog module > -- > > Key: OOZIE-3698 > URL: https://issues.apache.org/jira/browse/OOZIE-3698 > Project: Oozie > Issue Type: Sub-task > Components: build >Affects Versions: 5.2.1 >Reporter: Janos Makai >Priority: Major > Attachments: Oozie-oozie-sharelib-hcatalog-SpotBugs-issues.txt > > > This ticket is created for fixing or filtering the SpotBugs related issues in > Oozie's sharelib-hcatalog module. > The exact SpotBugs output is attached for this module. > There are currently 1 SpotBugs discovered issues in this module. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (OOZIE-3693) [examples] Fix current SpotBugs discovered issues in Oozie's examples module
[ https://issues.apache.org/jira/browse/OOZIE-3693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Janos Makai reassigned OOZIE-3693: -- Assignee: Janos Makai > [examples] Fix current SpotBugs discovered issues in Oozie's examples module > > > Key: OOZIE-3693 > URL: https://issues.apache.org/jira/browse/OOZIE-3693 > Project: Oozie > Issue Type: Sub-task > Components: build >Affects Versions: 5.2.1 >Reporter: Janos Makai >Assignee: Janos Makai >Priority: Major > Attachments: Oozie-examples-SpotBugs-issues.txt > > > This ticket is created for fixing or filtering the SpotBugs related issues in > Oozie's examples module. > The exact SpotBugs output is attached for this module. > There are currently 3 SpotBugs discovered issues in this module. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (OOZIE-3690) [server] Fix current SpotBugs discovered issues in Oozie's server module
[ https://issues.apache.org/jira/browse/OOZIE-3690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17651216#comment-17651216 ] Janos Makai commented on OOZIE-3690: Thank you, sure. I've opened OOZIE-3711 for it. > [server] Fix current SpotBugs discovered issues in Oozie's server module > > > Key: OOZIE-3690 > URL: https://issues.apache.org/jira/browse/OOZIE-3690 > Project: Oozie > Issue Type: Sub-task > Components: build >Affects Versions: 5.2.1 >Reporter: Janos Makai >Assignee: Janos Makai >Priority: Major > Attachments: OOZIE-3690-001.patch, Oozie-server-SpotBugs-issues.txt > > > This ticket is created for fixing or filtering the SpotBugs related issues in > Oozie's server module. > The exact SpotBugs output is attached for this module. > There are currently 10 SpotBugs discovered issues in this module. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (OOZIE-3711) Investigate heap size related issues occurring in PreCommit Oozie unit test runs
Janos Makai created OOZIE-3711: -- Summary: Investigate heap size related issues occurring in PreCommit Oozie unit test runs Key: OOZIE-3711 URL: https://issues.apache.org/jira/browse/OOZIE-3711 Project: Oozie Issue Type: Improvement Affects Versions: 5.2.1 Reporter: Janos Makai The goal of this Jira is to check and investigate the reason behind the recent heap size related issues (java.lang.OutOfMemoryError: Java heap space) in the Oozie unit test runs. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Comment Edited] (OOZIE-3479) Build pre-commit pipeline for pull requests.
[ https://issues.apache.org/jira/browse/OOZIE-3479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17651197#comment-17651197 ] Janos Makai edited comment on OOZIE-3479 at 12/22/22 9:40 AM: -- Thank you for the inputs [~gezapeti] and [~dionusos]. Acquiring and printing the outputs of the test results and surefire reports (of the failed tests at least) should be straight forward according to my research. Regarding the release log, I agree on executing it automatically based on the PR's title, this should be done easily in the container as well by modifying the release log file, and then by using a GitHub push action. was (Author: jmakai): Thank you for the inputs [~gezapeti] and [~dionusos]. Acquiring and printing the outputs of the test results and surefire reports (of the failed ones at least) should be straight forward according to my research. Regarding the release log, I agree on executing it automatically based on the PR's title, this should be done easily in the container as well by modifying the release log file, and then by using a GitHub push action. > Build pre-commit pipeline for pull requests. > > > Key: OOZIE-3479 > URL: https://issues.apache.org/jira/browse/OOZIE-3479 > Project: Oozie > Issue Type: Task >Reporter: Gézapeti >Priority: Major > > I think it would be great to accept pull requests for Oozie as well. > We should build the pre-commit system out first to see how it works for us > before updating the site and changing the rules. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Comment Edited] (OOZIE-3479) Build pre-commit pipeline for pull requests.
[ https://issues.apache.org/jira/browse/OOZIE-3479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17651197#comment-17651197 ] Janos Makai edited comment on OOZIE-3479 at 12/22/22 9:40 AM: -- Thank you for the inputs [~gezapeti] and [~dionusos]. Acquiring and printing the outputs of the test results and surefire reports (of the failed ones at least) should be straight forward according to my research. Regarding the release log, I agree on executing it automatically based on the PR's title, this should be done easily in the container as well by modifying the release log file, and then by using a GitHub push action. was (Author: jmakai): Thank you for the inputs [~gezapeti] and [~dionusos]. Acquiring and printing the outputs of the test results and surefire reports (for the failed ones at least) should be straight forward according to my research. Regarding the release log, I agree on executing it automatically based on the PR's title, this should be done easily in the container as well by modifying the release log file, and then by using a GitHub push action. > Build pre-commit pipeline for pull requests. > > > Key: OOZIE-3479 > URL: https://issues.apache.org/jira/browse/OOZIE-3479 > Project: Oozie > Issue Type: Task >Reporter: Gézapeti >Priority: Major > > I think it would be great to accept pull requests for Oozie as well. > We should build the pre-commit system out first to see how it works for us > before updating the site and changing the rules. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (OOZIE-3479) Build pre-commit pipeline for pull requests.
[ https://issues.apache.org/jira/browse/OOZIE-3479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17651197#comment-17651197 ] Janos Makai commented on OOZIE-3479: Thank you for the inputs [~gezapeti] and [~dionusos]. Acquiring and printing the outputs of the test results and surefire reports (for the failed ones at least) should be straight forward according to my research. Regarding the release log, I agree on executing it automatically based on the PR's title, this should be done easily in the container as well by modifying the release log file, and then by using a GitHub push action. > Build pre-commit pipeline for pull requests. > > > Key: OOZIE-3479 > URL: https://issues.apache.org/jira/browse/OOZIE-3479 > Project: Oozie > Issue Type: Task >Reporter: Gézapeti >Priority: Major > > I think it would be great to accept pull requests for Oozie as well. > We should build the pre-commit system out first to see how it works for us > before updating the site and changing the rules. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (OOZIE-3690) [server] Fix current SpotBugs discovered issues in Oozie's server module
[ https://issues.apache.org/jira/browse/OOZIE-3690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17651179#comment-17651179 ] Janos Makai commented on OOZIE-3690: Hi [~dionusos] could you please relaunch the PreCommit tests? Looks like we were hitting heap space issues at the time of the PreCommit run. Thank you > [server] Fix current SpotBugs discovered issues in Oozie's server module > > > Key: OOZIE-3690 > URL: https://issues.apache.org/jira/browse/OOZIE-3690 > Project: Oozie > Issue Type: Sub-task > Components: build >Affects Versions: 5.2.1 >Reporter: Janos Makai >Assignee: Janos Makai >Priority: Major > Attachments: OOZIE-3690-001.patch, Oozie-server-SpotBugs-issues.txt > > > This ticket is created for fixing or filtering the SpotBugs related issues in > Oozie's server module. > The exact SpotBugs output is attached for this module. > There are currently 10 SpotBugs discovered issues in this module. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (OOZIE-3692) [sharelib-spark] Fix current SpotBugs discovered issues in Oozie's sharelib-spark module
[ https://issues.apache.org/jira/browse/OOZIE-3692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Janos Makai reassigned OOZIE-3692: -- Assignee: Janos Makai > [sharelib-spark] Fix current SpotBugs discovered issues in Oozie's > sharelib-spark module > > > Key: OOZIE-3692 > URL: https://issues.apache.org/jira/browse/OOZIE-3692 > Project: Oozie > Issue Type: Sub-task > Components: build >Affects Versions: 5.2.1 >Reporter: Janos Makai >Assignee: Janos Makai >Priority: Major > Attachments: Oozie-oozie-sharelib-spark-SpotBugs-issues.txt > > > This ticket is created for fixing or filtering the SpotBugs related issues in > Oozie's sharelib-spark module. > The exact SpotBugs output is attached for this module. > There are currently 4 SpotBugs discovered issues in this module. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (OOZIE-3690) [server] Fix current SpotBugs discovered issues in Oozie's server module
[ https://issues.apache.org/jira/browse/OOZIE-3690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Janos Makai reassigned OOZIE-3690: -- Assignee: Janos Makai > [server] Fix current SpotBugs discovered issues in Oozie's server module > > > Key: OOZIE-3690 > URL: https://issues.apache.org/jira/browse/OOZIE-3690 > Project: Oozie > Issue Type: Sub-task > Components: build >Affects Versions: 5.2.1 >Reporter: Janos Makai >Assignee: Janos Makai >Priority: Major > Attachments: Oozie-server-SpotBugs-issues.txt > > > This ticket is created for fixing or filtering the SpotBugs related issues in > Oozie's server module. > The exact SpotBugs output is attached for this module. > There are currently 10 SpotBugs discovered issues in this module. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (OOZIE-3672) Fix current SpotBugs warnings
[ https://issues.apache.org/jira/browse/OOZIE-3672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Janos Makai reassigned OOZIE-3672: -- Assignee: (was: Janos Makai) > Fix current SpotBugs warnings > - > > Key: OOZIE-3672 > URL: https://issues.apache.org/jira/browse/OOZIE-3672 > Project: Oozie > Issue Type: Task >Affects Versions: 5.2.1 >Reporter: Janos Makai >Priority: Major > > Currently the SpotBugs tool indicates hundreds of warnings across almost all > of the Oozie modules. > This Jira serves as a similar initiative like it was tracked under OOZIE-1987 > but this is aimed for the current warnings and potential bugs. > Additional information: Due to there are ~800 issues discovered by SpotBugs > in Oozie's core module, we have decided to create separate tickets for each > type of issues. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (OOZIE-3672) Fix current SpotBugs warnings
[ https://issues.apache.org/jira/browse/OOZIE-3672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Janos Makai reassigned OOZIE-3672: -- Assignee: Janos Makai > Fix current SpotBugs warnings > - > > Key: OOZIE-3672 > URL: https://issues.apache.org/jira/browse/OOZIE-3672 > Project: Oozie > Issue Type: Task >Affects Versions: 5.2.1 >Reporter: Janos Makai >Assignee: Janos Makai >Priority: Major > > Currently the SpotBugs tool indicates hundreds of warnings across almost all > of the Oozie modules. > This Jira serves as a similar initiative like it was tracked under OOZIE-1987 > but this is aimed for the current warnings and potential bugs. > Additional information: Due to there are ~800 issues discovered by SpotBugs > in Oozie's core module, we have decided to create separate tickets for each > type of issues. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (OOZIE-3696) [sharelib-git] Fix current SpotBugs discovered issues in Oozie's sharelib-git module
[ https://issues.apache.org/jira/browse/OOZIE-3696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17650472#comment-17650472 ] Janos Makai commented on OOZIE-3696: Hi [~dionusos] , could you please check the unit test results? Also, please use .toLowerCase() in your patch set Thanks. > [sharelib-git] Fix current SpotBugs discovered issues in Oozie's sharelib-git > module > > > Key: OOZIE-3696 > URL: https://issues.apache.org/jira/browse/OOZIE-3696 > Project: Oozie > Issue Type: Sub-task > Components: build >Affects Versions: 5.2.1 >Reporter: Janos Makai >Assignee: Dénes Bodó >Priority: Major > Attachments: OOZIE-3696-001.patch, > Oozie-oozie-sharelib-git-SpotBugs-issues.txt > > > This ticket is created for fixing or filtering the SpotBugs related issues in > Oozie's sharelib-git module. > The exact SpotBugs output is attached for this module. > There are currently 2 SpotBugs discovered issues in this module. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (OOZIE-3479) Build pre-commit pipeline for pull requests.
[ https://issues.apache.org/jira/browse/OOZIE-3479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17650413#comment-17650413 ] Janos Makai commented on OOZIE-3479: Using GitHub actions seems to have great potential and I think that could help simplifying the PreCommit system for the Github PRs. With that we can easily execute the build process and running the unit tests as @Gézapeti mentioned. The main concern @Gézapeti raised is that there might be a problem acquiring the changes of the PR. One direction that looks promising is that we could gather all the changed files corresponding to a PR using a changed files gathering GitHub action, and then we could launch our scripts against those files in the container (e.g. SpotBugs, RAW patch analysis, backward compatibility checks, etc..) Examples of this can be found here: [LINK|https://github.com/marketplace/actions/changed-files] and here: [LINK|https://docs.github.com/en/actions/examples/using-scripts-to-test-your-code-on-a-runner] What do you think of this approach? > Build pre-commit pipeline for pull requests. > > > Key: OOZIE-3479 > URL: https://issues.apache.org/jira/browse/OOZIE-3479 > Project: Oozie > Issue Type: Task >Reporter: Gézapeti >Priority: Major > > I think it would be great to accept pull requests for Oozie as well. > We should build the pre-commit system out first to see how it works for us > before updating the site and changing the rules. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (OOZIE-3695) [sharelib-hive2] Fix current SpotBugs discovered issues in Oozie's sharelib-hive2 module
[ https://issues.apache.org/jira/browse/OOZIE-3695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Janos Makai updated OOZIE-3695: --- Attachment: Oozie-oozie-sharelib-hive2-SpotBugs-issues.txt > [sharelib-hive2] Fix current SpotBugs discovered issues in Oozie's > sharelib-hive2 module > > > Key: OOZIE-3695 > URL: https://issues.apache.org/jira/browse/OOZIE-3695 > Project: Oozie > Issue Type: Sub-task > Components: build >Affects Versions: 5.2.1 >Reporter: Janos Makai >Priority: Major > Attachments: Oozie-oozie-sharelib-hive2-SpotBugs-issues.txt > > > This ticket is created for fixing or filtering the SpotBugs related issues in > Oozie's sharelib-hive2 module. > The exact SpotBugs output is attached for this module. > There are currently 4 SpotBugs discovered issues in this module. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (OOZIE-3672) Fix current SpotBugs warnings
[ https://issues.apache.org/jira/browse/OOZIE-3672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Janos Makai updated OOZIE-3672: --- Description: Currently the SpotBugs tool indicates hundreds of warnings across almost all of the Oozie modules. This Jira serves as a similar initiative like it was tracked under OOZIE-1987 but this is aimed for the current warnings and potential bugs. Additional information: Due to there are ~800 issues discovered by SpotBugs in Oozie's core module, we have decided to create separate tickets for each type of issues. was: Currently the SpotBugs tool indicates hundreds of warnings across almost all of the Oozie modules. This Jira serves as a similar initiative like it was tracked under OOZIE-1987 but this is aimed for the current warnings and potential bugs. > Fix current SpotBugs warnings > - > > Key: OOZIE-3672 > URL: https://issues.apache.org/jira/browse/OOZIE-3672 > Project: Oozie > Issue Type: Task >Affects Versions: 5.2.1 >Reporter: Janos Makai >Priority: Major > > Currently the SpotBugs tool indicates hundreds of warnings across almost all > of the Oozie modules. > This Jira serves as a similar initiative like it was tracked under OOZIE-1987 > but this is aimed for the current warnings and potential bugs. > Additional information: Due to there are ~800 issues discovered by SpotBugs > in Oozie's core module, we have decided to create separate tickets for each > type of issues. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (OOZIE-3710) [core] Fix current SpotBugs discovered [Dodgy code] issues in Oozie's core module
Janos Makai created OOZIE-3710: -- Summary: [core] Fix current SpotBugs discovered [Dodgy code] issues in Oozie's core module Key: OOZIE-3710 URL: https://issues.apache.org/jira/browse/OOZIE-3710 Project: Oozie Issue Type: Sub-task Components: build Affects Versions: 5.2.1 Reporter: Janos Makai This ticket is created for fixing or filtering the SpotBugs related [Dodgy code] issues in Oozie's core module. There are currently 235 SpotBugs discovered issues in this module with this type of issue. In order to filter them, using the SpotBugs GUI is advised. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (OOZIE-3709) [core] Fix current SpotBugs discovered [Security] issues in Oozie's core module
Janos Makai created OOZIE-3709: -- Summary: [core] Fix current SpotBugs discovered [Security] issues in Oozie's core module Key: OOZIE-3709 URL: https://issues.apache.org/jira/browse/OOZIE-3709 Project: Oozie Issue Type: Sub-task Components: build Affects Versions: 5.2.1 Reporter: Janos Makai This ticket is created for fixing or filtering the SpotBugs related [Security] issues in Oozie's core module. There are currently 236 SpotBugs discovered issues in this module with this type of issue. In order to filter them, using the SpotBugs GUI is advised. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (OOZIE-3708) [core] Fix current SpotBugs discovered [Performance] issues in Oozie's core module
Janos Makai created OOZIE-3708: -- Summary: [core] Fix current SpotBugs discovered [Performance] issues in Oozie's core module Key: OOZIE-3708 URL: https://issues.apache.org/jira/browse/OOZIE-3708 Project: Oozie Issue Type: Sub-task Components: build Affects Versions: 5.2.1 Reporter: Janos Makai This ticket is created for fixing or filtering the SpotBugs related [Performance] issues in Oozie's core module. There are currently 103 SpotBugs discovered issues in this module with this type of issue. In order to filter them, using the SpotBugs GUI is advised. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (OOZIE-3707) [core] Fix current SpotBugs discovered [Multithreaded correctness] issues in Oozie's core module
Janos Makai created OOZIE-3707: -- Summary: [core] Fix current SpotBugs discovered [Multithreaded correctness] issues in Oozie's core module Key: OOZIE-3707 URL: https://issues.apache.org/jira/browse/OOZIE-3707 Project: Oozie Issue Type: Sub-task Components: build Affects Versions: 5.2.1 Reporter: Janos Makai This ticket is created for fixing or filtering the SpotBugs related [Multithreaded correctness] issues in Oozie's core module. There are currently 19 SpotBugs discovered issues in this module with this type of issue. In order to filter them, using the SpotBugs GUI is advised. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (OOZIE-3706) [core] Fix current SpotBugs discovered [Malicious code vulnerability] issues in Oozie's core module
Janos Makai created OOZIE-3706: -- Summary: [core] Fix current SpotBugs discovered [Malicious code vulnerability] issues in Oozie's core module Key: OOZIE-3706 URL: https://issues.apache.org/jira/browse/OOZIE-3706 Project: Oozie Issue Type: Sub-task Components: build Affects Versions: 5.2.1 Reporter: Janos Makai This ticket is created for fixing or filtering the SpotBugs related [Malicious code vulnerability] issues in Oozie's core module. There are currently 60 SpotBugs discovered issues in this module with this type of issue. In order to filter them, using the SpotBugs GUI is advised. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (OOZIE-3705) [core] Fix current SpotBugs discovered [Internationalization] issues in Oozie's core module
Janos Makai created OOZIE-3705: -- Summary: [core] Fix current SpotBugs discovered [Internationalization] issues in Oozie's core module Key: OOZIE-3705 URL: https://issues.apache.org/jira/browse/OOZIE-3705 Project: Oozie Issue Type: Sub-task Components: build Affects Versions: 5.2.1 Reporter: Janos Makai This ticket is created for fixing or filtering the SpotBugs related [Internationalization] issues in Oozie's core module. There are currently 39 SpotBugs discovered issues in this module with this type of issue. In order to filter them, using the SpotBugs GUI is advised. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (OOZIE-3704) [core] Fix current SpotBugs discovered [Experimental] issues in Oozie's core module
Janos Makai created OOZIE-3704: -- Summary: [core] Fix current SpotBugs discovered [Experimental] issues in Oozie's core module Key: OOZIE-3704 URL: https://issues.apache.org/jira/browse/OOZIE-3704 Project: Oozie Issue Type: Sub-task Components: build Affects Versions: 5.2.1 Reporter: Janos Makai This ticket is created for fixing or filtering the SpotBugs related [Experimental] issues in Oozie's core module. There are currently 4 SpotBugs discovered issues in this module with this type of issue. In order to filter them, using the SpotBugs GUI is advised. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (OOZIE-3703) [core] Fix current SpotBugs discovered [Bad practice] issues in Oozie's core module
Janos Makai created OOZIE-3703: -- Summary: [core] Fix current SpotBugs discovered [Bad practice] issues in Oozie's core module Key: OOZIE-3703 URL: https://issues.apache.org/jira/browse/OOZIE-3703 Project: Oozie Issue Type: Sub-task Components: build Affects Versions: 5.2.1 Reporter: Janos Makai This ticket is created for fixing or filtering the SpotBugs related [Bad practice] issues in Oozie's core module. There are currently 68 SpotBugs discovered issues in this module with this type of issue. In order to filter them, using the SpotBugs GUI is advised. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (OOZIE-3702) [core] Fix current SpotBugs discovered [Correctness] issues in Oozie's core module
Janos Makai created OOZIE-3702: -- Summary: [core] Fix current SpotBugs discovered [Correctness] issues in Oozie's core module Key: OOZIE-3702 URL: https://issues.apache.org/jira/browse/OOZIE-3702 Project: Oozie Issue Type: Sub-task Components: build Affects Versions: 5.2.1 Reporter: Janos Makai This ticket is created for fixing or filtering the SpotBugs related [Correctness] issues in Oozie's core module. There are currently 18 SpotBugs discovered issues in this module with this type of issue. In order to filter them, using the SpotBugs GUI is advised. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (OOZIE-3701) [client] Fix current SpotBugs discovered issues in Oozie's client module
Janos Makai created OOZIE-3701: -- Summary: [client] Fix current SpotBugs discovered issues in Oozie's client module Key: OOZIE-3701 URL: https://issues.apache.org/jira/browse/OOZIE-3701 Project: Oozie Issue Type: Sub-task Components: build Affects Versions: 5.2.1 Reporter: Janos Makai Attachments: Oozie-client-SpotBugs-issues.txt This ticket is created for fixing or filtering the SpotBugs related issues in Oozie's client module. The exact SpotBugs output is attached for this module. There are currently 80 SpotBugs discovered issues in this module. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (OOZIE-3700) [fluent-job-api] Fix current SpotBugs discovered issues in Oozie's fluent-job-api module
Janos Makai created OOZIE-3700: -- Summary: [fluent-job-api] Fix current SpotBugs discovered issues in Oozie's fluent-job-api module Key: OOZIE-3700 URL: https://issues.apache.org/jira/browse/OOZIE-3700 Project: Oozie Issue Type: Sub-task Components: build Affects Versions: 5.2.1 Reporter: Janos Makai Attachments: Oozie-fluent-job-api-SpotBugs-issues.txt This ticket is created for fixing or filtering the SpotBugs related issues in Oozie's fluent-job-api module. The exact SpotBugs output is attached for this module. There are currently 2 SpotBugs discovered issues in this module. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (OOZIE-3699) [sharelib-oozie] Fix current SpotBugs discovered issues in Oozie's sharelib-oozie module
Janos Makai created OOZIE-3699: -- Summary: [sharelib-oozie] Fix current SpotBugs discovered issues in Oozie's sharelib-oozie module Key: OOZIE-3699 URL: https://issues.apache.org/jira/browse/OOZIE-3699 Project: Oozie Issue Type: Sub-task Components: build Affects Versions: 5.2.1 Reporter: Janos Makai Attachments: Oozie-oozie-sharelib-oozie-SpotBugs-issues.txt This ticket is created for fixing or filtering the SpotBugs related issues in Oozie's sharelib-oozie module. The exact SpotBugs output is attached for this module. There are currently 38 SpotBugs discovered issues in this module. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (OOZIE-3698) [sharelib-hcatalog] Fix current SpotBugs discovered issues in Oozie's sharelib-hcatalog module
Janos Makai created OOZIE-3698: -- Summary: [sharelib-hcatalog] Fix current SpotBugs discovered issues in Oozie's sharelib-hcatalog module Key: OOZIE-3698 URL: https://issues.apache.org/jira/browse/OOZIE-3698 Project: Oozie Issue Type: Sub-task Components: build Affects Versions: 5.2.1 Reporter: Janos Makai This ticket is created for fixing or filtering the SpotBugs related issues in Oozie's sharelib-hcatalog module. The exact SpotBugs output is attached for this module. There are currently 1 SpotBugs discovered issues in this module. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (OOZIE-3697) [sharelib-pig] Fix current SpotBugs discovered issues in Oozie's sharelib-pig module
Janos Makai created OOZIE-3697: -- Summary: [sharelib-pig] Fix current SpotBugs discovered issues in Oozie's sharelib-pig module Key: OOZIE-3697 URL: https://issues.apache.org/jira/browse/OOZIE-3697 Project: Oozie Issue Type: Sub-task Components: build Affects Versions: 5.2.1 Reporter: Janos Makai Attachments: Oozie-oozie-sharelib-pig-SpotBugs-issues.txt This ticket is created for fixing or filtering the SpotBugs related issues in Oozie's sharelib-pig module. The exact SpotBugs output is attached for this module. There are currently 6 SpotBugs discovered issues in this module. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (OOZIE-3696) [sharelib-git] Fix current SpotBugs discovered issues in Oozie's sharelib-git module
Janos Makai created OOZIE-3696: -- Summary: [sharelib-git] Fix current SpotBugs discovered issues in Oozie's sharelib-git module Key: OOZIE-3696 URL: https://issues.apache.org/jira/browse/OOZIE-3696 Project: Oozie Issue Type: Sub-task Components: build Affects Versions: 5.2.1 Reporter: Janos Makai Attachments: Oozie-oozie-sharelib-git-SpotBugs-issues.txt This ticket is created for fixing or filtering the SpotBugs related issues in Oozie's sharelib-git module. The exact SpotBugs output is attached for this module. There are currently 2 SpotBugs discovered issues in this module. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (OOZIE-3695) [sharelib-hive2] Fix current SpotBugs discovered issues in Oozie's sharelib-hive2 module
Janos Makai created OOZIE-3695: -- Summary: [sharelib-hive2] Fix current SpotBugs discovered issues in Oozie's sharelib-hive2 module Key: OOZIE-3695 URL: https://issues.apache.org/jira/browse/OOZIE-3695 Project: Oozie Issue Type: Sub-task Components: build Affects Versions: 5.2.1 Reporter: Janos Makai This ticket is created for fixing or filtering the SpotBugs related issues in Oozie's sharelib-hive2 module. The exact SpotBugs output is attached for this module. There are currently 4 SpotBugs discovered issues in this module. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (OOZIE-3694) [sharelib-hive] Fix current SpotBugs discovered issues in Oozie's sharelib-hive module
Janos Makai created OOZIE-3694: -- Summary: [sharelib-hive] Fix current SpotBugs discovered issues in Oozie's sharelib-hive module Key: OOZIE-3694 URL: https://issues.apache.org/jira/browse/OOZIE-3694 Project: Oozie Issue Type: Sub-task Components: build Affects Versions: 5.2.1 Reporter: Janos Makai Attachments: Oozie-oozie-sharelib-hive-SpotBugs-issues.txt This ticket is created for fixing or filtering the SpotBugs related issues in Oozie's sharelib-hive module. The exact SpotBugs output is attached for this module. There are currently 2 SpotBugs discovered issues in this module. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (OOZIE-3693) [examples] Fix current SpotBugs discovered issues in Oozie's examples module
[ https://issues.apache.org/jira/browse/OOZIE-3693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Janos Makai updated OOZIE-3693: --- Description: This ticket is created for fixing or filtering the SpotBugs related issues in Oozie's examples module. The exact SpotBugs output is attached for this module. There are currently 3 SpotBugs discovered issues in this module. was: This ticket is created for fixing or filtering the SpotBugs related issues in Oozie's examples module. The exact SpotBugs output is attached for this module. > [examples] Fix current SpotBugs discovered issues in Oozie's examples module > > > Key: OOZIE-3693 > URL: https://issues.apache.org/jira/browse/OOZIE-3693 > Project: Oozie > Issue Type: Sub-task > Components: build >Affects Versions: 5.2.1 >Reporter: Janos Makai >Priority: Major > Attachments: Oozie-examples-SpotBugs-issues.txt > > > This ticket is created for fixing or filtering the SpotBugs related issues in > Oozie's examples module. > The exact SpotBugs output is attached for this module. > There are currently 3 SpotBugs discovered issues in this module. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (OOZIE-3692) [sharelib-spark] Fix current SpotBugs discovered issues in Oozie's sharelib-spark module
[ https://issues.apache.org/jira/browse/OOZIE-3692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Janos Makai updated OOZIE-3692: --- Description: This ticket is created for fixing or filtering the SpotBugs related issues in Oozie's sharelib-spark module. The exact SpotBugs output is attached for this module. There are currently 4 SpotBugs discovered issues in this module. was: This ticket is created for fixing or filtering the SpotBugs related issues in Oozie's sharelib-spark module. The exact SpotBugs output is attached for this module. > [sharelib-spark] Fix current SpotBugs discovered issues in Oozie's > sharelib-spark module > > > Key: OOZIE-3692 > URL: https://issues.apache.org/jira/browse/OOZIE-3692 > Project: Oozie > Issue Type: Sub-task > Components: build >Affects Versions: 5.2.1 >Reporter: Janos Makai >Priority: Major > Attachments: Oozie-oozie-sharelib-spark-SpotBugs-issues.txt > > > This ticket is created for fixing or filtering the SpotBugs related issues in > Oozie's sharelib-spark module. > The exact SpotBugs output is attached for this module. > There are currently 4 SpotBugs discovered issues in this module. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (OOZIE-3690) [server] Fix current SpotBugs discovered issues in Oozie's server module
[ https://issues.apache.org/jira/browse/OOZIE-3690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Janos Makai updated OOZIE-3690: --- Description: This ticket is created for fixing or filtering the SpotBugs related issues in Oozie's server module. The exact SpotBugs output is attached for this module. There are currently 10 SpotBugs discovered issues in this module. was: This ticket is created for fixing or filtering the SpotBugs related issues in Oozie's server module. The exact SpotBugs output is attached for this module. > [server] Fix current SpotBugs discovered issues in Oozie's server module > > > Key: OOZIE-3690 > URL: https://issues.apache.org/jira/browse/OOZIE-3690 > Project: Oozie > Issue Type: Sub-task > Components: build >Affects Versions: 5.2.1 >Reporter: Janos Makai >Priority: Major > Attachments: Oozie-server-SpotBugs-issues.txt > > > This ticket is created for fixing or filtering the SpotBugs related issues in > Oozie's server module. > The exact SpotBugs output is attached for this module. > There are currently 10 SpotBugs discovered issues in this module. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (OOZIE-3691) [tools] Fix current SpotBugs discovered issues in Oozie's tools module
[ https://issues.apache.org/jira/browse/OOZIE-3691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Janos Makai updated OOZIE-3691: --- Description: This ticket is created for fixing or filtering the SpotBugs related issues in Oozie's tools module. The exact SpotBugs output is attached for this module. There are currently 108 SpotBugs discovered issues in this module. was: This ticket is created for fixing or filtering the SpotBugs related issues in Oozie's tools module. The exact SpotBugs output is attached for this module. > [tools] Fix current SpotBugs discovered issues in Oozie's tools module > -- > > Key: OOZIE-3691 > URL: https://issues.apache.org/jira/browse/OOZIE-3691 > Project: Oozie > Issue Type: Sub-task > Components: build >Affects Versions: 5.2.1 >Reporter: Janos Makai >Priority: Major > Attachments: Oozie-tools-SpotBugs-issues.txt > > > This ticket is created for fixing or filtering the SpotBugs related issues in > Oozie's tools module. > The exact SpotBugs output is attached for this module. > There are currently 108 SpotBugs discovered issues in this module. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (OOZIE-3692) [sharelib-spark] Fix current SpotBugs discovered issues in Oozie's sharelib-spark module
[ https://issues.apache.org/jira/browse/OOZIE-3692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Janos Makai updated OOZIE-3692: --- Affects Version/s: 5.2.1 > [sharelib-spark] Fix current SpotBugs discovered issues in Oozie's > sharelib-spark module > > > Key: OOZIE-3692 > URL: https://issues.apache.org/jira/browse/OOZIE-3692 > Project: Oozie > Issue Type: Sub-task > Components: build >Affects Versions: 5.2.1 >Reporter: Janos Makai >Priority: Major > Attachments: Oozie-oozie-sharelib-spark-SpotBugs-issues.txt > > > This ticket is created for fixing or filtering the SpotBugs related issues in > Oozie's sharelib-spark module. > The exact SpotBugs output is attached for this module. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (OOZIE-3693) [examples] Fix current SpotBugs discovered issues in Oozie's examples module
Janos Makai created OOZIE-3693: -- Summary: [examples] Fix current SpotBugs discovered issues in Oozie's examples module Key: OOZIE-3693 URL: https://issues.apache.org/jira/browse/OOZIE-3693 Project: Oozie Issue Type: Sub-task Components: build Affects Versions: 5.2.1 Reporter: Janos Makai Attachments: Oozie-examples-SpotBugs-issues.txt This ticket is created for fixing or filtering the SpotBugs related issues in Oozie's examples module. The exact SpotBugs output is attached for this module. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (OOZIE-3691) [tools] Fix current SpotBugs discovered issues in Oozie's tools module
[ https://issues.apache.org/jira/browse/OOZIE-3691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Janos Makai updated OOZIE-3691: --- Affects Version/s: 5.2.1 > [tools] Fix current SpotBugs discovered issues in Oozie's tools module > -- > > Key: OOZIE-3691 > URL: https://issues.apache.org/jira/browse/OOZIE-3691 > Project: Oozie > Issue Type: Sub-task > Components: build >Affects Versions: 5.2.1 >Reporter: Janos Makai >Priority: Major > Attachments: Oozie-tools-SpotBugs-issues.txt > > > This ticket is created for fixing or filtering the SpotBugs related issues in > Oozie's tools module. > The exact SpotBugs output is attached for this module. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (OOZIE-3692) [sharelib-spark] Fix current SpotBugs discovered issues in Oozie's sharelib-spark module
Janos Makai created OOZIE-3692: -- Summary: [sharelib-spark] Fix current SpotBugs discovered issues in Oozie's sharelib-spark module Key: OOZIE-3692 URL: https://issues.apache.org/jira/browse/OOZIE-3692 Project: Oozie Issue Type: Sub-task Components: build Reporter: Janos Makai Attachments: Oozie-oozie-sharelib-spark-SpotBugs-issues.txt This ticket is created for fixing or filtering the SpotBugs related issues in Oozie's sharelib-spark module. The exact SpotBugs output is attached for this module. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (OOZIE-3691) [tools] Fix current SpotBugs discovered issues in Oozie's tools module
Janos Makai created OOZIE-3691: -- Summary: [tools] Fix current SpotBugs discovered issues in Oozie's tools module Key: OOZIE-3691 URL: https://issues.apache.org/jira/browse/OOZIE-3691 Project: Oozie Issue Type: Sub-task Components: build Reporter: Janos Makai Attachments: Oozie-tools-SpotBugs-issues.txt This ticket is created for fixing or filtering the SpotBugs related issues in Oozie's tools module. The exact SpotBugs output is attached for this module. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (OOZIE-3690) [server] Fix current SpotBugs discovered issues in Oozie's server module
Janos Makai created OOZIE-3690: -- Summary: [server] Fix current SpotBugs discovered issues in Oozie's server module Key: OOZIE-3690 URL: https://issues.apache.org/jira/browse/OOZIE-3690 Project: Oozie Issue Type: Sub-task Components: build Affects Versions: 5.2.1 Reporter: Janos Makai Attachments: Oozie-server-SpotBugs-issues.txt This ticket is created for fixing or filtering the SpotBugs related issues in Oozie's server module. The exact SpotBugs output is attached for this module. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (OOZIE-3642) Investigate if Oozie can utilize Hadoop's Configuration change
[ https://issues.apache.org/jira/browse/OOZIE-3642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17649686#comment-17649686 ] Janos Makai commented on OOZIE-3642: Unfortunately HADOOP-18021 got merged to Hadoop 3, so we cannot use that method until we make sure Oozie works with Hadoop 3 (OOZIE-2973) > Investigate if Oozie can utilize Hadoop's Configuration change > -- > > Key: OOZIE-3642 > URL: https://issues.apache.org/jira/browse/OOZIE-3642 > Project: Oozie > Issue Type: Task >Affects Versions: 5.1.0 >Reporter: Dénes Bodó >Priority: Major > > There was a recent change in Hadoop's Configuration class which made > substituteVars method public from private. As this is an interface change > which could broke Oozie build HADOOP-18021 is created to fix the issue came > with the interface change. > The task is to check if we can eliminate > [https://github.com/apache/oozie/blob/8dfdb35b640686781e89f63460018eaa65d8e455/core/src/main/java/org/apache/oozie/util/XConfiguration.java#L186] > and use the one from org.apache.hadoop.conf.Configuration. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (OOZIE-3614) Do not launched Oozie Job in YARN.
[ https://issues.apache.org/jira/browse/OOZIE-3614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17649370#comment-17649370 ] Janos Makai commented on OOZIE-3614: Hi [~mellowH], There can be several reasons why the AM initialization is not complete in time, hence investigating the AM logs would be useful to troubleshoot further your problem. Can you please attach it in case you are still hitting this issue? > Do not launched Oozie Job in YARN. > -- > > Key: OOZIE-3614 > URL: https://issues.apache.org/jira/browse/OOZIE-3614 > Project: Oozie > Issue Type: Bug > Components: action >Affects Versions: 5.2.0 > Environment: +Yarn-site.xml+ > +!image-2020-12-04-13-30-10-175.png!+ > ++ > +hdfs-site.xml+ > +!image-2020-12-04-13-31-06-477.png!+ > ++ > +core-site.xml+ > +!image-2020-12-04-13-31-33-583.png!+ >Reporter: hanminseok >Priority: Critical > Attachments: image-2020-12-04-13-07-23-123.png, > image-2020-12-04-13-08-10-996.png, image-2020-12-04-13-08-44-789.png, > image-2020-12-04-13-10-36-296.png, image-2020-12-04-13-15-34-529.png, > image-2020-12-04-13-17-41-938.png, image-2020-12-04-13-18-25-169.png, > image-2020-12-04-13-27-58-707.png, image-2020-12-04-13-30-10-175.png, > image-2020-12-04-13-31-06-477.png, image-2020-12-04-13-31-33-583.png > > > *did not launched oozie job in YARN.* > *YARN Application stuck 'Accepted status'.* > *I dont know why* > > --- > I was installed Oozie5.2.0 > Hadoop is Singe Node Cluster. > Hadoop and oozie were configured using Docker. > Hardware Spec is 16G ram, 4 core cpu. > - > Trying to run examples provided by Oozie. > The shell was executed among examples. > h1. *Oozie* > 1. job.properties > !image-2020-12-04-13-08-10-996.png|width=649,height=96! > > 2. workflow.xml > !image-2020-12-04-13-08-44-789.png|width=758,height=435! > 3. Job Start > $oozie job -config job.properties -run > !image-2020-12-04-13-10-36-296.png|width=591,height=260! > - > h1. YARN Resource Manager > * > -- Stuck [Submitted Status]* > !image-2020-12-04-13-15-34-529.png|width=787,height=359! > > *-Resource Manager Info* > !image-2020-12-04-13-17-41-938.png! > > *-Log* > [Node manager] > !image-2020-12-04-13-27-58-707.png|width=1508,height=328! > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (OOZIE-3554) Add asf.yaml to git repo
[
https://issues.apache.org/jira/browse/OOZIE-3554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17649360#comment-17649360
]
Janos Makai commented on OOZIE-3554:
Similarly to LUCENE-8986, I've created a .asf.yaml file. [~asalamon74] /
[~dionusos] could you please the created .asf.yaml file? Thank you.
> Add asf.yaml to git repo
>
>
> Key: OOZIE-3554
> URL: https://issues.apache.org/jira/browse/OOZIE-3554
> Project: Oozie
> Issue Type: Sub-task
>Affects Versions: trunk
>Reporter: Andras Salamon
>Assignee: Janos Makai
>Priority: Major
> Attachments: OOZIE-3554-001.patch
>
>
> Add an {{asf.yaml}} file to the git repo similarly to LUCENE-8986. This will
> be useful for the pull-request handling as well.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Comment Edited] (OOZIE-3554) Add asf.yaml to git repo
[
https://issues.apache.org/jira/browse/OOZIE-3554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17649360#comment-17649360
]
Janos Makai edited comment on OOZIE-3554 at 12/19/22 2:45 PM:
--
Similarly to LUCENE-8986, I've created a .asf.yaml file. [~asalamon74] /
[~dionusos] could you please review the created .asf.yaml file? Thank you.
was (Author: jmakai):
Similarly to LUCENE-8986, I've created a .asf.yaml file. [~asalamon74] /
[~dionusos] could you please the created .asf.yaml file? Thank you.
> Add asf.yaml to git repo
>
>
> Key: OOZIE-3554
> URL: https://issues.apache.org/jira/browse/OOZIE-3554
> Project: Oozie
> Issue Type: Sub-task
>Affects Versions: trunk
>Reporter: Andras Salamon
>Assignee: Janos Makai
>Priority: Major
> Attachments: OOZIE-3554-001.patch
>
>
> Add an {{asf.yaml}} file to the git repo similarly to LUCENE-8986. This will
> be useful for the pull-request handling as well.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Assigned] (OOZIE-3554) Add asf.yaml to git repo
[
https://issues.apache.org/jira/browse/OOZIE-3554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Janos Makai reassigned OOZIE-3554:
--
Assignee: Janos Makai
> Add asf.yaml to git repo
>
>
> Key: OOZIE-3554
> URL: https://issues.apache.org/jira/browse/OOZIE-3554
> Project: Oozie
> Issue Type: Sub-task
>Affects Versions: trunk
>Reporter: Andras Salamon
>Assignee: Janos Makai
>Priority: Major
>
> Add an {{asf.yaml}} file to the git repo similarly to LUCENE-8986. This will
> be useful for the pull-request handling as well.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (OOZIE-3689) Remove usage of commons-httpclient due to EOL
[ https://issues.apache.org/jira/browse/OOZIE-3689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17649193#comment-17649193 ] Janos Makai commented on OOZIE-3689: Thank you [~dionusos] > Remove usage of commons-httpclient due to EOL > - > > Key: OOZIE-3689 > URL: https://issues.apache.org/jira/browse/OOZIE-3689 > Project: Oozie > Issue Type: Improvement >Affects Versions: 5.2.1 >Reporter: Janos Makai >Assignee: Janos Makai >Priority: Major > Fix For: 5.3.0 > > Attachments: OOZIE-3689-001.patch > > > Currently usages of `commons-httpclient` can be found > `TestHAShareLibService`'s unit test, however, `commons-httpclient` is in EOL > state. > Oozie at this moment does not pull it in directly, but used from a transitive > dependency. > The goal of this ticket is to modify the test inside `TestHAShareLibService` > not to use `commons-httpclient`. > NOTE: this change only introduces a unit test change and does not contains > production change. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (OOZIE-3689) Remove usage of commons-httpclient due to EOL
[ https://issues.apache.org/jira/browse/OOZIE-3689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17648195#comment-17648195 ] Janos Makai commented on OOZIE-3689: Hi [~dionusos] could you please review the provided patch? The issues found by SpotBugs are unrelated to this change. > Remove usage of commons-httpclient due to EOL > - > > Key: OOZIE-3689 > URL: https://issues.apache.org/jira/browse/OOZIE-3689 > Project: Oozie > Issue Type: Improvement >Affects Versions: 5.2.1 >Reporter: Janos Makai >Assignee: Janos Makai >Priority: Major > Attachments: OOZIE-3689-001.patch > > > Currently usages of `commons-httpclient` can be found > `TestHAShareLibService`'s unit test, however, `commons-httpclient` is in EOL > state. > Oozie at this moment does not pull it in directly, but used from a transitive > dependency. > The goal of this ticket is to modify the test inside `TestHAShareLibService` > not to use `commons-httpclient`. > NOTE: this change only introduces a unit test change and does not contains > production change. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (OOZIE-3689) Remove usage of commons-httpclient due to EOL
[ https://issues.apache.org/jira/browse/OOZIE-3689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Janos Makai updated OOZIE-3689: --- Description: Currently usages of `commons-httpclient` can be found `TestHAShareLibService`'s unit test, however, `commons-httpclient` is in EOL state. Oozie at this moment does not pull it in directly, but used from a transitive dependency. The goal of this ticket is to modify the test inside `TestHAShareLibService` not to use `commons-httpclient`. NOTE: this change only introduces a unit test change and does not contains production change. was: Currently usages of `commons-httpclient` can be found `TestHAShareLibService`'s unit test, however, `commons-httpclient` is in EOL state and has CVE issue (CVE-2012-5783). Oozie at this moment does not pull it in directly, but used from a transitive dependency. The goal of this ticket is to modify the test inside `TestHAShareLibService` not to use `commons-httpclient`. > Remove usage of commons-httpclient due to EOL > - > > Key: OOZIE-3689 > URL: https://issues.apache.org/jira/browse/OOZIE-3689 > Project: Oozie > Issue Type: Improvement >Affects Versions: 5.2.1 >Reporter: Janos Makai >Assignee: Janos Makai >Priority: Major > Attachments: OOZIE-3689-001.patch > > > Currently usages of `commons-httpclient` can be found > `TestHAShareLibService`'s unit test, however, `commons-httpclient` is in EOL > state. > Oozie at this moment does not pull it in directly, but used from a transitive > dependency. > The goal of this ticket is to modify the test inside `TestHAShareLibService` > not to use `commons-httpclient`. > NOTE: this change only introduces a unit test change and does not contains > production change. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (OOZIE-3689) Remove usage of commons-httpclient due to EOL
[ https://issues.apache.org/jira/browse/OOZIE-3689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Janos Makai updated OOZIE-3689: --- Summary: Remove usage of commons-httpclient due to EOL (was: Remove usage of commons-httpclient due to EOL and CVEs (CVE-2012-5783)) > Remove usage of commons-httpclient due to EOL > - > > Key: OOZIE-3689 > URL: https://issues.apache.org/jira/browse/OOZIE-3689 > Project: Oozie > Issue Type: Improvement >Affects Versions: 5.2.1 >Reporter: Janos Makai >Assignee: Janos Makai >Priority: Major > Attachments: OOZIE-3689-001.patch > > > Currently usages of `commons-httpclient` can be found > `TestHAShareLibService`'s unit test, however, `commons-httpclient` is in EOL > state and has CVE issue (CVE-2012-5783). > Oozie at this moment does not pull it in directly, but used from a transitive > dependency. > The goal of this ticket is to modify the test inside `TestHAShareLibService` > not to use `commons-httpclient`. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (OOZIE-3689) Remove usage of commons-httpclient due to EOL and CVEs (CVE-2012-5783)
Janos Makai created OOZIE-3689: -- Summary: Remove usage of commons-httpclient due to EOL and CVEs (CVE-2012-5783) Key: OOZIE-3689 URL: https://issues.apache.org/jira/browse/OOZIE-3689 Project: Oozie Issue Type: Improvement Affects Versions: 5.2.1 Reporter: Janos Makai Assignee: Janos Makai Currently usages of `commons-httpclient` can be found `TestHAShareLibService`'s unit test, however, `commons-httpclient` is in EOL state and has CVE issue (CVE-2012-5783). Oozie at this moment does not pull it in directly, but used from a transitive dependency. The goal of this ticket is to modify the test inside `TestHAShareLibService` not to use `commons-httpclient`. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (OOZIE-3687) Fix Oozie client always using the current system username instead the one specified by the user (e.g.: via kerberos or via explicit basic authentication)
[ https://issues.apache.org/jira/browse/OOZIE-3687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Janos Makai updated OOZIE-3687: --- Attachment: OOZIE-3687-002.patch > Fix Oozie client always using the current system username instead the one > specified by the user (e.g.: via kerberos or via explicit basic > authentication) > - > > Key: OOZIE-3687 > URL: https://issues.apache.org/jira/browse/OOZIE-3687 > Project: Oozie > Issue Type: Bug >Affects Versions: 5.2.1 >Reporter: Janos Makai >Assignee: Janos Makai >Priority: Major > Attachments: OOZIE-3687-001.patch, OOZIE-3687-002.patch > > > Oozie client relies on user.name system property now which is not secure in a > kerberised environment. This has to be changed. > A bug in Oozie CLI causes the Workflow to be launched in the name of the > current Unix user even if Kerberos authentication is used with a ticket for a > different user. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (OOZIE-3688) Introduce retry mechanism when starting embedded servlet containers in unit tests
[ https://issues.apache.org/jira/browse/OOZIE-3688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17647573#comment-17647573 ] Janos Makai commented on OOZIE-3688: Many thanks for the review! :) > Introduce retry mechanism when starting embedded servlet containers in unit > tests > - > > Key: OOZIE-3688 > URL: https://issues.apache.org/jira/browse/OOZIE-3688 > Project: Oozie > Issue Type: Bug >Affects Versions: 5.2.1 >Reporter: Janos Makai >Assignee: Janos Makai >Priority: Major > Fix For: 5.3.0 > > Attachments: OOZIE-3688-001.patch > > > In the unit tests which uses the embedded servlet container an intermittent > issue can occur without this change, because those tests start the container > on a specific port, which is available at the start of the unit tests. > However, in some corner cases, it can happen that the port is being available > at the start of the test, but some process occupies that port meanwhile, > leading to a `java.net.BindException: Address already in use` exception > during the unit test's execution. > The goal of this ticket is resolve this by introducing a retry mechanism when > starting the embedded servlet container. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (OOZIE-3688) Introduce retry mechanism when starting embedded servlet containers in unit tests
[ https://issues.apache.org/jira/browse/OOZIE-3688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Janos Makai updated OOZIE-3688: --- Description: In the unit tests which uses the embedded servlet container an intermittent issue can occur without this change, because those tests start the container on a specific port, which is available at the start of the unit tests. However, in some corner cases, it can happen that the port is being available at the start of the test, but some process occupies that port meanwhile, leading to a `java.net.BindException: Address already in use` exception during the unit test's execution. The goal of this ticket is resolve this by introducing a retry mechanism when starting the embedded servlet container. was: In the unit tests which uses the embedded servlet container an intermittent issue can occur without this change, because those tests start the container in a specific port, which is available at the start of the unit tests. However, in some corner cases, it can happen that the port is being available at the start of the test, but some process occupies that port meanwhile, leading to a `java.net.BindException: Address already in use` exception during the unit test's execution. The goal of this ticket is resolve this by introducing a retry mechanism when starting the embedded servlet container. > Introduce retry mechanism when starting embedded servlet containers in unit > tests > - > > Key: OOZIE-3688 > URL: https://issues.apache.org/jira/browse/OOZIE-3688 > Project: Oozie > Issue Type: Bug >Affects Versions: 5.2.1 >Reporter: Janos Makai >Assignee: Janos Makai >Priority: Major > Attachments: OOZIE-3688-001.patch > > > In the unit tests which uses the embedded servlet container an intermittent > issue can occur without this change, because those tests start the container > on a specific port, which is available at the start of the unit tests. > However, in some corner cases, it can happen that the port is being available > at the start of the test, but some process occupies that port meanwhile, > leading to a `java.net.BindException: Address already in use` exception > during the unit test's execution. > The goal of this ticket is resolve this by introducing a retry mechanism when > starting the embedded servlet container. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Comment Edited] (OOZIE-3688) Introduce retry mechanism when starting embedded servlet containers in unit tests
[ https://issues.apache.org/jira/browse/OOZIE-3688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17647541#comment-17647541 ] Janos Makai edited comment on OOZIE-3688 at 12/14/22 3:32 PM: -- Hello [~dionusos], could you please review my patch? The issues found by SpotBugs are unrelated to this change. was (Author: jmakai): Hello [~dionusos], could you please my patch? The issues found by SpotBugs are unrelated to this change. > Introduce retry mechanism when starting embedded servlet containers in unit > tests > - > > Key: OOZIE-3688 > URL: https://issues.apache.org/jira/browse/OOZIE-3688 > Project: Oozie > Issue Type: Bug >Affects Versions: 5.2.1 >Reporter: Janos Makai >Assignee: Janos Makai >Priority: Major > Attachments: OOZIE-3688-001.patch > > > In the unit tests which uses the embedded servlet container an intermittent > issue can occur without this change, because those tests start the container > in a specific port, which is available at the start of the unit tests. > However, in some corner cases, it can happen that the port is being available > at the start of the test, but some process occupies that port meanwhile, > leading to a `java.net.BindException: Address already in use` exception > during the unit test's execution. > The goal of this ticket is resolve this by introducing a retry mechanism when > starting the embedded servlet container. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (OOZIE-3688) Introduce retry mechanism when starting embedded servlet containers in unit tests
[ https://issues.apache.org/jira/browse/OOZIE-3688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17647541#comment-17647541 ] Janos Makai commented on OOZIE-3688: Hello [~dionusos], could you please my patch? The issues found by SpotBugs are unrelated to this change. > Introduce retry mechanism when starting embedded servlet containers in unit > tests > - > > Key: OOZIE-3688 > URL: https://issues.apache.org/jira/browse/OOZIE-3688 > Project: Oozie > Issue Type: Bug >Affects Versions: 5.2.1 >Reporter: Janos Makai >Assignee: Janos Makai >Priority: Major > Attachments: OOZIE-3688-001.patch > > > In the unit tests which uses the embedded servlet container an intermittent > issue can occur without this change, because those tests start the container > in a specific port, which is available at the start of the unit tests. > However, in some corner cases, it can happen that the port is being available > at the start of the test, but some process occupies that port meanwhile, > leading to a `java.net.BindException: Address already in use` exception > during the unit test's execution. > The goal of this ticket is resolve this by introducing a retry mechanism when > starting the embedded servlet container. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (OOZIE-3688) Introduce retry mechanism when starting embedded servlet containers in unit tests
Janos Makai created OOZIE-3688: -- Summary: Introduce retry mechanism when starting embedded servlet containers in unit tests Key: OOZIE-3688 URL: https://issues.apache.org/jira/browse/OOZIE-3688 Project: Oozie Issue Type: Bug Affects Versions: 5.2.1 Reporter: Janos Makai Assignee: Janos Makai In the unit tests which uses the embedded servlet container an intermittent issue can occur without this change, because those tests start the container in a specific port, which is available at the start of the unit tests. However, in some corner cases, it can happen that the port is being available at the start of the test, but some process occupies that port meanwhile, leading to a `java.net.BindException: Address already in use` exception during the unit test's execution. The goal of this ticket is resolve this by introducing a retry mechanism when starting the embedded servlet container. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (OOZIE-3687) Fix Oozie client always using the current system username instead the one specified by the user (e.g.: via kerberos or via explicit basic authentication)
[ https://issues.apache.org/jira/browse/OOZIE-3687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Janos Makai reassigned OOZIE-3687: -- Assignee: Janos Makai > Fix Oozie client always using the current system username instead the one > specified by the user (e.g.: via kerberos or via explicit basic > authentication) > - > > Key: OOZIE-3687 > URL: https://issues.apache.org/jira/browse/OOZIE-3687 > Project: Oozie > Issue Type: Bug >Affects Versions: 5.2.1 >Reporter: Janos Makai >Assignee: Janos Makai >Priority: Major > > Oozie client relies on user.name system property now which is not secure in a > kerberised environment. This has to be changed. > A bug in Oozie CLI causes the Workflow to be launched in the name of the > current Unix user even if Kerberos authentication is used with a ticket for a > different user. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (OOZIE-3686) Repair flaky testRetryConsoleUrlForked unit test
[
https://issues.apache.org/jira/browse/OOZIE-3686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17645184#comment-17645184
]
Janos Makai commented on OOZIE-3686:
Thank you :)
> Repair flaky testRetryConsoleUrlForked unit test
>
>
> Key: OOZIE-3686
> URL: https://issues.apache.org/jira/browse/OOZIE-3686
> Project: Oozie
> Issue Type: Bug
>Affects Versions: 5.2.1
>Reporter: Janos Makai
>Assignee: Janos Makai
>Priority: Major
> Fix For: 5.3.0
>
> Attachments: OOZIE-3686-001.patch
>
>
> testRetryConsoleUrlForked can fail periodically and can be reproduced
> locally. The following exception is thrown:
> {code:java}
> java.lang.NullPointerException
> at
> org.apache.oozie.command.wf.TestWorkflowActionRetryInfoXCommand.validateRetryConsoleUrl(TestWorkflowActionRetryInfoXCommand.java:192)
> at
> org.apache.oozie.command.wf.TestWorkflowActionRetryInfoXCommand.testRetryConsoleUrlForked(TestWorkflowActionRetryInfoXCommand.java:126)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at junit.framework.TestCase.runTest(TestCase.java:177)
> at junit.framework.TestCase.runBare(TestCase.java:142)
> at junit.framework.TestResult$1.protect(TestResult.java:122)
> at junit.framework.TestResult.runProtected(TestResult.java:142)
> at junit.framework.TestResult.run(TestResult.java:125)
> at junit.framework.TestCase.run(TestCase.java:130)
> at junit.framework.TestSuite.runTest(TestSuite.java:241)
> at junit.framework.TestSuite.run(TestSuite.java:236)
> at
> org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:90)
> at org.junit.runners.Suite.runChild(Suite.java:128)
> at org.junit.runners.Suite.runChild(Suite.java:27)
> at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
> at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
> at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
> at org.junit.runners.Suite.runChild(Suite.java:128)
> at org.junit.runners.Suite.runChild(Suite.java:27)
> at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
> at
> org.apache.maven.surefire.junitcore.pc.Scheduler$1.run(Scheduler.java:410)
> at
> org.apache.maven.surefire.junitcore.pc.InvokerStrategy.schedule(InvokerStrategy.java:54)
> at
> org.apache.maven.surefire.junitcore.pc.Scheduler.schedule(Scheduler.java:367)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
> at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
> at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
> at
> org.apache.maven.surefire.junitcore.pc.ParallelComputerBuilder$PC$1.run(ParallelComputerBuilder.java:590)
> at org.apache.maven.surefire.junitcore.JUnitCore.run(JUnitCore.java:55)
> at
> org.apache.maven.surefire.junitcore.JUnitCoreWrapper.createRequestAndRun(JUnitCoreWrapper.java:137)
> at
> org.apache.maven.surefire.junitcore.JUnitCoreWrapper.executeEager(JUnitCoreWrapper.java:107)
> at
> org.apache.maven.surefire.junitcore.JUnitCoreWrapper.execute(JUnitCoreWrapper.java:83)
> at
> org.apache.maven.surefire.junitcore.JUnitCoreWrapper.execute(JUnitCoreWrapper.java:75)
> at
> org.apache.maven.surefire.junitcore.JUnitCoreProvider.invoke(JUnitCoreProvider.java:158)
> at
> org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:384)
> at
> org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:345)
> at
> org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:126)
> at
> org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:418)
> {code}
> The goal of this ticket is to prevent the above NPE to happen by adding a
> sleep which lets Oozie finish its back end DB synchronisation.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (OOZIE-3685) Repair flaky testCoordActionInputCheckXCommandUniqueness unit test
[
https://issues.apache.org/jira/browse/OOZIE-3685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17645183#comment-17645183
]
Janos Makai commented on OOZIE-3685:
Many thanks :)
> Repair flaky testCoordActionInputCheckXCommandUniqueness unit test
> --
>
> Key: OOZIE-3685
> URL: https://issues.apache.org/jira/browse/OOZIE-3685
> Project: Oozie
> Issue Type: Bug
>Reporter: Janos Makai
>Assignee: Janos Makai
>Priority: Major
> Fix For: 5.3.0
>
> Attachments: OOZIE-3685-001.patch
>
>
> org.apache.oozie.command.coord.TestCoordActionInputCheckXCommand#testCoordActionInputCheckXCommandUniqueness
> unit test is flaky.
> This unit test failure can be reproduced easily using the following command:
> {code:java}
> mvn clean package test
> -Dtest=org.apache.oozie.command.coord.TestCoordActionInputCheckXCommand#testCoordActionInputCheckXCommandUniqueness
> {code}
> The goal of this Jira is to eliminate this intermittent behaviour.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (OOZIE-3684) Migrate to commons-lang3 again
[ https://issues.apache.org/jira/browse/OOZIE-3684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17645181#comment-17645181 ] Janos Makai commented on OOZIE-3684: Thank you :) > Migrate to commons-lang3 again > -- > > Key: OOZIE-3684 > URL: https://issues.apache.org/jira/browse/OOZIE-3684 > Project: Oozie > Issue Type: Improvement > Components: core >Affects Versions: 5.2.1 >Reporter: Janos Makai >Assignee: Janos Makai >Priority: Major > Fix For: 5.3.0 > > Attachments: OOZIE-3684-001.patch > > > There was already a commons-lang 2->3 upgrade in scope of OOZIE-3446, but > OOZIE-3492 brought it back again to: > org.apache.oozie.action.hadoop.SparkActionExecutor: > https://github.com/apache/oozie/blob/3a932ad1cd7e294576b03c48bbd28e691ad6f58b/core/src/main/java/org/apache/oozie/action/hadoop/SparkActionExecutor.java#L22 > The goal of this Jira is to change this import to the commons-lang3 version > one to make Oozie fully migrated to commons-lang3 again. > _NOTE_: the commons-lang 2 version is available via transitive dependency. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (OOZIE-3687) Fix Oozie client always using the current system username instead the one specified by the user (e.g.: via kerberos or via explicit basic authentication)
Janos Makai created OOZIE-3687: -- Summary: Fix Oozie client always using the current system username instead the one specified by the user (e.g.: via kerberos or via explicit basic authentication) Key: OOZIE-3687 URL: https://issues.apache.org/jira/browse/OOZIE-3687 Project: Oozie Issue Type: Bug Affects Versions: 5.2.1 Reporter: Janos Makai Oozie client relies on user.name system property now which is not secure in a kerberised environment. This has to be changed. A bug in Oozie CLI causes the Workflow to be launched in the name of the current Unix user even if Kerberos authentication is used with a ticket for a different user. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (OOZIE-3685) Repair flaky testCoordActionInputCheckXCommandUniqueness unit test
[
https://issues.apache.org/jira/browse/OOZIE-3685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17644874#comment-17644874
]
Janos Makai commented on OOZIE-3685:
Sure, Dénes.
During my investigation, I've found that if we use the queue() method for
launching callables, then we do not have much control over the execution and
outcome of them, and it could be different on different machines so it's safe
to say that it's not deterministic.
However, queueSerial() can be used instead in order to achieve deterministic
outcome, which then leads to eliminating the flakiness of the test.
Furthermore, I've made several testing locally to prove this.
I hope this helped you to understand the nature of this issue and feel free to
ask if something is not clear.
> Repair flaky testCoordActionInputCheckXCommandUniqueness unit test
> --
>
> Key: OOZIE-3685
> URL: https://issues.apache.org/jira/browse/OOZIE-3685
> Project: Oozie
> Issue Type: Bug
>Reporter: Janos Makai
>Assignee: Janos Makai
>Priority: Major
> Attachments: OOZIE-3685-001.patch
>
>
> org.apache.oozie.command.coord.TestCoordActionInputCheckXCommand#testCoordActionInputCheckXCommandUniqueness
> unit test is flaky.
> This unit test failure can be reproduced easily using the following command:
> {code:java}
> mvn clean package test
> -Dtest=org.apache.oozie.command.coord.TestCoordActionInputCheckXCommand#testCoordActionInputCheckXCommandUniqueness
> {code}
> The goal of this Jira is to eliminate this intermittent behaviour.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (OOZIE-3686) Repair flaky testRetryConsoleUrlForked unit test
Janos Makai created OOZIE-3686:
--
Summary: Repair flaky testRetryConsoleUrlForked unit test
Key: OOZIE-3686
URL: https://issues.apache.org/jira/browse/OOZIE-3686
Project: Oozie
Issue Type: Bug
Affects Versions: 5.2.1
Reporter: Janos Makai
Assignee: Janos Makai
testRetryConsoleUrlForked can fail periodically and can be reproduced locally.
The following exception is thrown:
{code:java}
java.lang.NullPointerException
at
org.apache.oozie.command.wf.TestWorkflowActionRetryInfoXCommand.validateRetryConsoleUrl(TestWorkflowActionRetryInfoXCommand.java:192)
at
org.apache.oozie.command.wf.TestWorkflowActionRetryInfoXCommand.testRetryConsoleUrlForked(TestWorkflowActionRetryInfoXCommand.java:126)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at junit.framework.TestCase.runTest(TestCase.java:177)
at junit.framework.TestCase.runBare(TestCase.java:142)
at junit.framework.TestResult$1.protect(TestResult.java:122)
at junit.framework.TestResult.runProtected(TestResult.java:142)
at junit.framework.TestResult.run(TestResult.java:125)
at junit.framework.TestCase.run(TestCase.java:130)
at junit.framework.TestSuite.runTest(TestSuite.java:241)
at junit.framework.TestSuite.run(TestSuite.java:236)
at
org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:90)
at org.junit.runners.Suite.runChild(Suite.java:128)
at org.junit.runners.Suite.runChild(Suite.java:27)
at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
at org.junit.runners.Suite.runChild(Suite.java:128)
at org.junit.runners.Suite.runChild(Suite.java:27)
at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
at
org.apache.maven.surefire.junitcore.pc.Scheduler$1.run(Scheduler.java:410)
at
org.apache.maven.surefire.junitcore.pc.InvokerStrategy.schedule(InvokerStrategy.java:54)
at
org.apache.maven.surefire.junitcore.pc.Scheduler.schedule(Scheduler.java:367)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
at
org.apache.maven.surefire.junitcore.pc.ParallelComputerBuilder$PC$1.run(ParallelComputerBuilder.java:590)
at org.apache.maven.surefire.junitcore.JUnitCore.run(JUnitCore.java:55)
at
org.apache.maven.surefire.junitcore.JUnitCoreWrapper.createRequestAndRun(JUnitCoreWrapper.java:137)
at
org.apache.maven.surefire.junitcore.JUnitCoreWrapper.executeEager(JUnitCoreWrapper.java:107)
at
org.apache.maven.surefire.junitcore.JUnitCoreWrapper.execute(JUnitCoreWrapper.java:83)
at
org.apache.maven.surefire.junitcore.JUnitCoreWrapper.execute(JUnitCoreWrapper.java:75)
at
org.apache.maven.surefire.junitcore.JUnitCoreProvider.invoke(JUnitCoreProvider.java:158)
at
org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:384)
at
org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:345)
at
org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:126)
at
org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:418)
{code}
The goal of this ticket is to prevent the above NPE to happen by adding a sleep
which lets Oozie finish its back end DB synchronisation.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (OOZIE-3685) Repair flaky testCoordActionInputCheckXCommandUniqueness unit test
Janos Makai created OOZIE-3685:
--
Summary: Repair flaky testCoordActionInputCheckXCommandUniqueness
unit test
Key: OOZIE-3685
URL: https://issues.apache.org/jira/browse/OOZIE-3685
Project: Oozie
Issue Type: Bug
Reporter: Janos Makai
Assignee: Janos Makai
org.apache.oozie.command.coord.TestCoordActionInputCheckXCommand#testCoordActionInputCheckXCommandUniqueness
unit test is flaky.
This unit test failure can be reproduced easily using the following command:
{code:java}
mvn clean package test
-Dtest=org.apache.oozie.command.coord.TestCoordActionInputCheckXCommand#testCoordActionInputCheckXCommandUniqueness
{code}
The goal of this Jira is to eliminate this intermittent behaviour.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (OOZIE-3682) Prepare ssh-wrapper script to handle callback specific arguments with quotes
[ https://issues.apache.org/jira/browse/OOZIE-3682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17644429#comment-17644429 ] Janos Makai commented on OOZIE-3682: Sounds good, thank yu! > Prepare ssh-wrapper script to handle callback specific arguments with quotes > > > Key: OOZIE-3682 > URL: https://issues.apache.org/jira/browse/OOZIE-3682 > Project: Oozie > Issue Type: Bug >Affects Versions: 5.2.1 >Reporter: Janos Makai >Assignee: Janos Makai >Priority: Major > Fix For: 5.3.0 > > Attachments: OOZIE-3682-001.patch, OOZIE-3682-002.patch > > > In case someone runs an SSH action which connects from Oozie server (linux) > to an AIX host, then the ssh-wrapper process is showing up on the remote > host, but it hangs forever. > This happens because the quotes around the callback URL are being retained in > the arguments taken by ssh-base and sent to ssh-wrapper, causing an infinite > loop. > The goal of this ticket is to track the fix of this issue in the callback > mechanism. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (OOZIE-3684) Migrate to commons-lang3 again
[ https://issues.apache.org/jira/browse/OOZIE-3684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Janos Makai updated OOZIE-3684: --- Issue Type: Improvement (was: Bug) > Migrate to commons-lang3 again > -- > > Key: OOZIE-3684 > URL: https://issues.apache.org/jira/browse/OOZIE-3684 > Project: Oozie > Issue Type: Improvement > Components: core >Affects Versions: 5.2.1 >Reporter: Janos Makai >Assignee: Janos Makai >Priority: Major > > There was already a commons-lang 2->3 upgrade in scope of OOZIE-3446, but > OOZIE-3492 brought it back again to: > org.apache.oozie.action.hadoop.SparkActionExecutor: > https://github.com/apache/oozie/blob/3a932ad1cd7e294576b03c48bbd28e691ad6f58b/core/src/main/java/org/apache/oozie/action/hadoop/SparkActionExecutor.java#L22 > The goal of this Jira is to change this import to the commons-lang3 version > one to make Oozie fully migrated to commons-lang3 again. > _NOTE_: the commons-lang 2 version is available via transitive dependency. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (OOZIE-3684) Migrate to commons-lang3 again
Janos Makai created OOZIE-3684: -- Summary: Migrate to commons-lang3 again Key: OOZIE-3684 URL: https://issues.apache.org/jira/browse/OOZIE-3684 Project: Oozie Issue Type: Bug Components: core Affects Versions: 5.2.1 Reporter: Janos Makai Assignee: Janos Makai There was already a commons-lang 2->3 upgrade in scope of OOZIE-3446, but OOZIE-3492 brought it back again to: org.apache.oozie.action.hadoop.SparkActionExecutor: https://github.com/apache/oozie/blob/3a932ad1cd7e294576b03c48bbd28e691ad6f58b/core/src/main/java/org/apache/oozie/action/hadoop/SparkActionExecutor.java#L22 The goal of this Jira is to change this import to the commons-lang3 version one to make Oozie fully migrated to commons-lang3 again. _NOTE_: the commons-lang 2 version is available via transitive dependency. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (OOZIE-3682) Prepare ssh-wrapper script to handle callback specific arguments with quotes
[ https://issues.apache.org/jira/browse/OOZIE-3682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17644387#comment-17644387 ] Janos Makai commented on OOZIE-3682: [~dionusos] This change was tested on an AIX host and worked as expected. > Prepare ssh-wrapper script to handle callback specific arguments with quotes > > > Key: OOZIE-3682 > URL: https://issues.apache.org/jira/browse/OOZIE-3682 > Project: Oozie > Issue Type: Bug >Affects Versions: 5.2.1 >Reporter: Janos Makai >Assignee: Janos Makai >Priority: Major > Attachments: OOZIE-3682-001.patch, OOZIE-3682-002.patch > > > In case someone runs an SSH action which connects from Oozie server (linux) > to an AIX host, then the ssh-wrapper process is showing up on the remote > host, but it hangs forever. > This happens because the quotes around the callback URL are being retained in > the arguments taken by ssh-base and sent to ssh-wrapper, causing an infinite > loop. > The goal of this ticket is to track the fix of this issue in the callback > mechanism. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (OOZIE-3683) Fix intermittent unit test failure in testLoadOnRestart
[
https://issues.apache.org/jira/browse/OOZIE-3683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17644388#comment-17644388
]
Janos Makai commented on OOZIE-3683:
Thank you :)
> Fix intermittent unit test failure in testLoadOnRestart
>
>
> Key: OOZIE-3683
> URL: https://issues.apache.org/jira/browse/OOZIE-3683
> Project: Oozie
> Issue Type: Bug
> Components: core
>Affects Versions: 5.2.1
>Reporter: Janos Makai
>Assignee: Janos Makai
>Priority: Major
> Fix For: 5.3.0
>
> Attachments: OOZIE-3683-001.patch
>
>
> The following failure can happen occasionally in
> _org.apache.oozie.sla.TestSLACalculatorMemory.testLoadOnRestart_ unit test:
>
> {code:java}
> junit.framework.AssertionFailedError: expected:
> but was:
> at junit.framework.Assert.fail(Assert.java:57)
> at junit.framework.Assert.failNotEquals(Assert.java:329)
> at junit.framework.Assert.assertEquals(Assert.java:78)
> at junit.framework.Assert.assertEquals(Assert.java:86)
> at junit.framework.TestCase.assertEquals(TestCase.java:246)
> at
> org.apache.oozie.sla.TestSLACalculatorMemory.testLoadOnRestart(TestSLACalculatorMemory.java:230)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at junit.framework.TestCase.runTest(TestCase.java:177)
> at junit.framework.TestCase.runBare(TestCase.java:142)
> at junit.framework.TestResult$1.protect(TestResult.java:122)
> at junit.framework.TestResult.runProtected(TestResult.java:142)
> at junit.framework.TestResult.run(TestResult.java:125)
> at junit.framework.TestCase.run(TestCase.java:130)
> at junit.framework.TestSuite.runTest(TestSuite.java:241)
> at junit.framework.TestSuite.run(TestSuite.java:236)
> at
> org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:90)
> at org.junit.runners.Suite.runChild(Suite.java:128)
> at org.junit.runners.Suite.runChild(Suite.java:27)
> at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
> at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
> at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
> at org.junit.runners.Suite.runChild(Suite.java:128)
> at org.junit.runners.Suite.runChild(Suite.java:27)
> at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
> at
> org.apache.maven.surefire.junitcore.pc.Scheduler$1.run(Scheduler.java:410)
> at
> org.apache.maven.surefire.junitcore.pc.InvokerStrategy.schedule(InvokerStrategy.java:54)
> at
> org.apache.maven.surefire.junitcore.pc.Scheduler.schedule(Scheduler.java:367)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
> at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
> at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
> at
> org.apache.maven.surefire.junitcore.pc.ParallelComputerBuilder$PC$1.run(ParallelComputerBuilder.java:590)
> at org.apache.maven.surefire.junitcore.JUnitCore.run(JUnitCore.java:55)
> at
> org.apache.maven.surefire.junitcore.JUnitCoreWrapper.createRequestAndRun(JUnitCoreWrapper.java:137)
> at
> org.apache.maven.surefire.junitcore.JUnitCoreWrapper.executeEager(JUnitCoreWrapper.java:107)
> at
> org.apache.maven.surefire.junitcore.JUnitCoreWrapper.execute(JUnitCoreWrapper.java:83)
> at
> org.apache.maven.surefire.junitcore.JUnitCoreWrapper.execute(JUnitCoreWrapper.java:75)
> at
> org.apache.maven.surefire.junitcore.JUnitCoreProvider.invoke(JUnitCoreProvider.java:158)
> at
> org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:384)
> at
> org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:345)
> at
> org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:126)
> at
> org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:418)
> {code}
> The root cause of this is that the set
[jira] [Commented] (OOZIE-3679) Correct maximum wait time between database retry attempts property
[
https://issues.apache.org/jira/browse/OOZIE-3679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17644386#comment-17644386
]
Janos Makai commented on OOZIE-3679:
Many thanks. :)
> Correct maximum wait time between database retry attempts property
> --
>
> Key: OOZIE-3679
> URL: https://issues.apache.org/jira/browse/OOZIE-3679
> Project: Oozie
> Issue Type: Bug
>Reporter: Janos Makai
>Assignee: Janos Makai
>Priority: Major
> Fix For: 5.3.0
>
> Attachments: OOZIE-3679-001.patch, OOZIE-3679-002.patch,
> OOZIE-3679-003.patch
>
>
> Based on the oozie-default.xml:
> {code:xml}
>
> oozie.service.JPAService.retry.maximum-wait-time.ms
> 3
>
> Maximum wait time between database retry attempts.
>
>
> {code}
> it looks like we are missing a "{_}retry{_}" string before the maximum wait
> time between database retry attempts static string in Oozie's JPAService:
> {code:java}
> public static final String MAX_WAIT_TIME = CONF_PREFIX +
> "maximum-wait-time.ms";
> {code}
> {color:#172b4d}The goal of this change is to sync the config names between
> oozie-default.xml and Oozie but we should also support the old, invalid
> config name.{color}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (OOZIE-3683) Fix intermittent unit test failure in testLoadOnRestart
Janos Makai created OOZIE-3683:
--
Summary: Fix intermittent unit test failure in testLoadOnRestart
Key: OOZIE-3683
URL: https://issues.apache.org/jira/browse/OOZIE-3683
Project: Oozie
Issue Type: Bug
Components: core
Affects Versions: 5.2.1
Reporter: Janos Makai
Assignee: Janos Makai
The following failure can happen occasionally in
_org.apache.oozie.sla.TestSLACalculatorMemory.testLoadOnRestart_ unit test:
{code:java}
junit.framework.AssertionFailedError: expected:
but was:
at junit.framework.Assert.fail(Assert.java:57)
at junit.framework.Assert.failNotEquals(Assert.java:329)
at junit.framework.Assert.assertEquals(Assert.java:78)
at junit.framework.Assert.assertEquals(Assert.java:86)
at junit.framework.TestCase.assertEquals(TestCase.java:246)
at
org.apache.oozie.sla.TestSLACalculatorMemory.testLoadOnRestart(TestSLACalculatorMemory.java:230)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at junit.framework.TestCase.runTest(TestCase.java:177)
at junit.framework.TestCase.runBare(TestCase.java:142)
at junit.framework.TestResult$1.protect(TestResult.java:122)
at junit.framework.TestResult.runProtected(TestResult.java:142)
at junit.framework.TestResult.run(TestResult.java:125)
at junit.framework.TestCase.run(TestCase.java:130)
at junit.framework.TestSuite.runTest(TestSuite.java:241)
at junit.framework.TestSuite.run(TestSuite.java:236)
at
org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:90)
at org.junit.runners.Suite.runChild(Suite.java:128)
at org.junit.runners.Suite.runChild(Suite.java:27)
at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
at org.junit.runners.Suite.runChild(Suite.java:128)
at org.junit.runners.Suite.runChild(Suite.java:27)
at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
at
org.apache.maven.surefire.junitcore.pc.Scheduler$1.run(Scheduler.java:410)
at
org.apache.maven.surefire.junitcore.pc.InvokerStrategy.schedule(InvokerStrategy.java:54)
at
org.apache.maven.surefire.junitcore.pc.Scheduler.schedule(Scheduler.java:367)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
at
org.apache.maven.surefire.junitcore.pc.ParallelComputerBuilder$PC$1.run(ParallelComputerBuilder.java:590)
at org.apache.maven.surefire.junitcore.JUnitCore.run(JUnitCore.java:55)
at
org.apache.maven.surefire.junitcore.JUnitCoreWrapper.createRequestAndRun(JUnitCoreWrapper.java:137)
at
org.apache.maven.surefire.junitcore.JUnitCoreWrapper.executeEager(JUnitCoreWrapper.java:107)
at
org.apache.maven.surefire.junitcore.JUnitCoreWrapper.execute(JUnitCoreWrapper.java:83)
at
org.apache.maven.surefire.junitcore.JUnitCoreWrapper.execute(JUnitCoreWrapper.java:75)
at
org.apache.maven.surefire.junitcore.JUnitCoreProvider.invoke(JUnitCoreProvider.java:158)
at
org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:384)
at
org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:345)
at
org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:126)
at
org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:418)
{code}
The root cause of this is that the set expectedDuration can be too little,
leading to the restarted `SLACalculatorMemory` not getting updated properly in
_updateAllSlaStatus()_ after configuring the corresponding job before the
restart.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (OOZIE-3682) Prepare ssh-wrapper script to handle callback specific arguments with quotes
[ https://issues.apache.org/jira/browse/OOZIE-3682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17644248#comment-17644248 ] Janos Makai commented on OOZIE-3682: Hi [~dionusos], can you please check the results of the latest patch? Thank you > Prepare ssh-wrapper script to handle callback specific arguments with quotes > > > Key: OOZIE-3682 > URL: https://issues.apache.org/jira/browse/OOZIE-3682 > Project: Oozie > Issue Type: Bug >Affects Versions: 5.2.1 >Reporter: Janos Makai >Assignee: Janos Makai >Priority: Major > Attachments: OOZIE-3682-001.patch, OOZIE-3682-002.patch > > > In case someone runs an SSH action which connects from Oozie server (linux) > to an AIX host, then the ssh-wrapper process is showing up on the remote > host, but it hangs forever. > This happens because the quotes around the callback URL are being retained in > the arguments taken by ssh-base and sent to ssh-wrapper, causing an infinite > loop. > The goal of this ticket is to track the fix of this issue in the callback > mechanism. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (OOZIE-3679) Correct maximum wait time between database retry attempts property
[
https://issues.apache.org/jira/browse/OOZIE-3679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17644246#comment-17644246
]
Janos Makai commented on OOZIE-3679:
Hello [~dionusos],
could you please review the latest patch set and the precommit results? The
found bugs are unrelated to this change.
Thanks in advance.
> Correct maximum wait time between database retry attempts property
> --
>
> Key: OOZIE-3679
> URL: https://issues.apache.org/jira/browse/OOZIE-3679
> Project: Oozie
> Issue Type: Bug
>Reporter: Janos Makai
>Assignee: Janos Makai
>Priority: Major
> Attachments: OOZIE-3679-001.patch, OOZIE-3679-002.patch,
> OOZIE-3679-003.patch
>
>
> Based on the oozie-default.xml:
> {code:xml}
>
> oozie.service.JPAService.retry.maximum-wait-time.ms
> 3
>
> Maximum wait time between database retry attempts.
>
>
> {code}
> it looks like we are missing a "{_}retry{_}" string before the maximum wait
> time between database retry attempts static string in Oozie's JPAService:
> {code:java}
> public static final String MAX_WAIT_TIME = CONF_PREFIX +
> "maximum-wait-time.ms";
> {code}
> {color:#172b4d}The goal of this change is to sync the config names between
> oozie-default.xml and Oozie but we should also support the old, invalid
> config name.{color}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (OOZIE-3679) Correct maximum wait time between database retry attempts property
[
https://issues.apache.org/jira/browse/OOZIE-3679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Janos Makai updated OOZIE-3679:
---
Attachment: OOZIE-3679-003.patch
> Correct maximum wait time between database retry attempts property
> --
>
> Key: OOZIE-3679
> URL: https://issues.apache.org/jira/browse/OOZIE-3679
> Project: Oozie
> Issue Type: Bug
>Reporter: Janos Makai
>Assignee: Janos Makai
>Priority: Major
> Attachments: OOZIE-3679-001.patch, OOZIE-3679-002.patch,
> OOZIE-3679-003.patch
>
>
> Based on the oozie-default.xml:
> {code:xml}
>
> oozie.service.JPAService.retry.maximum-wait-time.ms
> 3
>
> Maximum wait time between database retry attempts.
>
>
> {code}
> it looks like we are missing a "{_}retry{_}" string before the maximum wait
> time between database retry attempts static string in Oozie's JPAService:
> {code:java}
> public static final String MAX_WAIT_TIME = CONF_PREFIX +
> "maximum-wait-time.ms";
> {code}
> {color:#172b4d}The goal of this change is to sync the config names between
> oozie-default.xml and Oozie but we should also support the old, invalid
> config name.{color}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (OOZIE-3681) Revise yarn.app and mapreduce property overrides in Oozie
[ https://issues.apache.org/jira/browse/OOZIE-3681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Janos Makai resolved OOZIE-3681. Resolution: Invalid > Revise yarn.app and mapreduce property overrides in Oozie > - > > Key: OOZIE-3681 > URL: https://issues.apache.org/jira/browse/OOZIE-3681 > Project: Oozie > Issue Type: Improvement >Affects Versions: 5.2.1 >Reporter: Janos Makai >Assignee: Janos Makai >Priority: Major > Attachments: OOZIE-3681-001.patch > > > According to the following documentation > [https://hadoop.apache.org/docs/r3.1.2/hadoop-yarn/hadoop-yarn-site/ResourceModel.html] > oozie.launcher.override.memory.mb and oozie.launcher.override.vcores are > outdated so the goal is to extend them with currently accepted property keys. > The current keys should be in the beginning of the list to have the highest > priority. Legacy overrides need to at end of the lists: > oozie.launcher.mapreduce.map.memory.mb & > oozie.launcher.mapreduce.map.cpu.vcores & > oozie.launcher.mapreduce.map.java.opts > The old keys should *not* be deleted to preserve backward compatibility of > Oozie and workflow configuration. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (OOZIE-3681) Revise yarn.app and mapreduce property overrides in Oozie
[ https://issues.apache.org/jira/browse/OOZIE-3681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17643274#comment-17643274 ] Janos Makai commented on OOZIE-3681: [~dionusos] thanks for the review, based on your suggestion I've checked the valid Hadoop 2.6.0 configuration properties, but they do not contain the ones that are being added with in this change, as unfortunately they're Hadoop 3 related. Therefore I'm closing this ticket. > Revise yarn.app and mapreduce property overrides in Oozie > - > > Key: OOZIE-3681 > URL: https://issues.apache.org/jira/browse/OOZIE-3681 > Project: Oozie > Issue Type: Improvement >Affects Versions: 5.2.1 >Reporter: Janos Makai >Assignee: Janos Makai >Priority: Major > Attachments: OOZIE-3681-001.patch > > > According to the following documentation > [https://hadoop.apache.org/docs/r3.1.2/hadoop-yarn/hadoop-yarn-site/ResourceModel.html] > oozie.launcher.override.memory.mb and oozie.launcher.override.vcores are > outdated so the goal is to extend them with currently accepted property keys. > The current keys should be in the beginning of the list to have the highest > priority. Legacy overrides need to at end of the lists: > oozie.launcher.mapreduce.map.memory.mb & > oozie.launcher.mapreduce.map.cpu.vcores & > oozie.launcher.mapreduce.map.java.opts > The old keys should *not* be deleted to preserve backward compatibility of > Oozie and workflow configuration. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (OOZIE-3682) Prepare ssh-wrapper script to handle callback specific arguments with quotes
[ https://issues.apache.org/jira/browse/OOZIE-3682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Janos Makai updated OOZIE-3682: --- Attachment: OOZIE-3682-002.patch > Prepare ssh-wrapper script to handle callback specific arguments with quotes > > > Key: OOZIE-3682 > URL: https://issues.apache.org/jira/browse/OOZIE-3682 > Project: Oozie > Issue Type: Bug >Affects Versions: 5.2.1 >Reporter: Janos Makai >Assignee: Janos Makai >Priority: Major > Attachments: OOZIE-3682-001.patch, OOZIE-3682-002.patch > > > In case someone runs an SSH action which connects from Oozie server (linux) > to an AIX host, then the ssh-wrapper process is showing up on the remote > host, but it hangs forever. > This happens because the quotes around the callback URL are being retained in > the arguments taken by ssh-base and sent to ssh-wrapper, causing an infinite > loop. > The goal of this ticket is to track the fix of this issue in the callback > mechanism. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (OOZIE-3606) Extend file system EL functions to use custom file system properties
[ https://issues.apache.org/jira/browse/OOZIE-3606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17643201#comment-17643201 ] Janos Makai commented on OOZIE-3606: Thank you [~dionusos] > Extend file system EL functions to use custom file system properties > > > Key: OOZIE-3606 > URL: https://issues.apache.org/jira/browse/OOZIE-3606 > Project: Oozie > Issue Type: Improvement > Components: core >Affects Versions: 5.2.0 >Reporter: Dénes Bodó >Assignee: Janos Makai >Priority: Major > Fix For: 5.3.0 > > Attachments: OOZIE-3606-001.patch, OOZIE-3606-002.patch, > OOZIE-3606-003.patch > > > In case we want to check file availability on abfss file system in scope of a > decision node and we have access to that file system only using a key file > which is password protected then currently we are blocked as we cannot allow > configuring hadoop.security.credstore.java-keystore-provider.password-file > globally. > This is why this ticket is created to propose a solution for this issue: > * allow special file system configurations only for file system EL functions > * let them configure globally via oozie-site.xml and for every workflow > independently -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (OOZIE-3677) Oozie should accept a keyStoreType and trustStoreType property in oozie-site.xml
[
https://issues.apache.org/jira/browse/OOZIE-3677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17643203#comment-17643203
]
Janos Makai commented on OOZIE-3677:
Thank you!
> Oozie should accept a keyStoreType and trustStoreType property in
> oozie-site.xml
>
>
> Key: OOZIE-3677
> URL: https://issues.apache.org/jira/browse/OOZIE-3677
> Project: Oozie
> Issue Type: Improvement
>Affects Versions: 5.2.1
>Reporter: Janos Makai
>Assignee: Janos Makai
>Priority: Major
> Fix For: 5.3.0
>
> Attachments: OOZIE-3677-001.patch
>
>
> In scope of making Oozie FIPS compliant, when TLS is enabled, the Oozie
> server expects a JKS KeyStore type even though the following Java system
> properties are being set:
> {code:java}
> -Djavax.net.ssl.keyStoreType=bcfks -Djavax.net.ssl.trustStoreType=bcfks{code}
> The goal of ticket is to make Oozie capable of setting _keyStoreType_ and
> _trustStoreType_ in oozie-site.xml if it's not already set by the user.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (OOZIE-3678) Reduce the number of NameNode access when starting the Yarn job
[
https://issues.apache.org/jira/browse/OOZIE-3678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17643200#comment-17643200
]
Janos Makai commented on OOZIE-3678:
Many thanks!
> Reduce the number of NameNode access when starting the Yarn job
> ---
>
> Key: OOZIE-3678
> URL: https://issues.apache.org/jira/browse/OOZIE-3678
> Project: Oozie
> Issue Type: Improvement
>Affects Versions: 5.2.1
>Reporter: Janos Makai
>Assignee: Janos Makai
>Priority: Major
> Fix For: 5.3.0
>
> Attachments: OOZIE-3678-001.patch, OOZIE-3678-002.patch
>
>
> Before Oozie starts the Yarn application it assembles the class path which's
> elements' paths are resolved with
> {code:java}
> remoteFS.resolvePath(p){code}
> which can cause slowness and high NameNode load.
> The intention is to make this path resolution configurable (ability to turn
> it on/off) via an Oozie configuration thus customers can turn it off when
> they experience job start slowness. Default value should result the new
> behaviour is active.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (OOZIE-3680) Add default value to custom configuration of all the supported file systems in Oozie
[ https://issues.apache.org/jira/browse/OOZIE-3680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17643198#comment-17643198 ] Janos Makai commented on OOZIE-3680: Thank you very much [~dionusos] :) > Add default value to custom configuration of all the supported file systems > in Oozie > > > Key: OOZIE-3680 > URL: https://issues.apache.org/jira/browse/OOZIE-3680 > Project: Oozie > Issue Type: Improvement >Affects Versions: 5.2.1 >Reporter: Janos Makai >Assignee: Janos Makai >Priority: Major > Fix For: 5.3.0 > > Attachments: OOZIE-3680-001.patch > > > The default supported file systems now include: hdfs, hftp, webhdfs, s3a, > abfs, abfss. > The goal of this ticket is to make sure the users don't have to set > explicitely the oozie.service.HadoopAccessorService.supported.filesystems > property if they want to use any of the above mentioned supported file > systems. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (OOZIE-3679) Correct maximum wait time between database retry attempts property
[
https://issues.apache.org/jira/browse/OOZIE-3679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Janos Makai updated OOZIE-3679:
---
Attachment: OOZIE-3679-002.patch
> Correct maximum wait time between database retry attempts property
> --
>
> Key: OOZIE-3679
> URL: https://issues.apache.org/jira/browse/OOZIE-3679
> Project: Oozie
> Issue Type: Bug
>Reporter: Janos Makai
>Assignee: Janos Makai
>Priority: Major
> Attachments: OOZIE-3679-001.patch, OOZIE-3679-002.patch
>
>
> Based on the oozie-default.xml:
> {code:xml}
>
> oozie.service.JPAService.retry.maximum-wait-time.ms
> 3
>
> Maximum wait time between database retry attempts.
>
>
> {code}
> it looks like we are missing a "{_}retry{_}" string before the maximum wait
> time between database retry attempts static string in Oozie's JPAService:
> {code:java}
> public static final String MAX_WAIT_TIME = CONF_PREFIX +
> "maximum-wait-time.ms";
> {code}
> {color:#172b4d}The goal of this change is to sync the config names between
> oozie-default.xml and Oozie but we should also support the old, invalid
> config name.{color}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (OOZIE-3682) Prepare ssh-wrapper script to handle callback specific arguments with quotes
Janos Makai created OOZIE-3682: -- Summary: Prepare ssh-wrapper script to handle callback specific arguments with quotes Key: OOZIE-3682 URL: https://issues.apache.org/jira/browse/OOZIE-3682 Project: Oozie Issue Type: Bug Affects Versions: 5.2.1 Reporter: Janos Makai Assignee: Janos Makai In case someone runs an SSH action which connects from Oozie server (linux) to an AIX host, then the ssh-wrapper process is showing up on the remote host, but it hangs forever. This happens because the quotes around the callback URL are being retained in the arguments taken by ssh-base and sent to ssh-wrapper, causing an infinite loop. The goal of this ticket is to track the fix of this issue in the callback mechanism. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (OOZIE-3681) Revise yarn.app and mapreduce property overrides in Oozie
Janos Makai created OOZIE-3681: -- Summary: Revise yarn.app and mapreduce property overrides in Oozie Key: OOZIE-3681 URL: https://issues.apache.org/jira/browse/OOZIE-3681 Project: Oozie Issue Type: Improvement Affects Versions: 5.2.1 Reporter: Janos Makai Assignee: Janos Makai According to the following documentation [https://hadoop.apache.org/docs/r3.1.2/hadoop-yarn/hadoop-yarn-site/ResourceModel.html] oozie.launcher.override.memory.mb and oozie.launcher.override.vcores are outdated so the goal is to extend them with currently accepted property keys. The current keys should be in the beginning of the list to have the highest priority. Legacy overrides need to at end of the lists: oozie.launcher.mapreduce.map.memory.mb & oozie.launcher.mapreduce.map.cpu.vcores & oozie.launcher.mapreduce.map.java.opts The old keys should *not* be deleted to preserve backward compatibility of Oozie and workflow configuration. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (OOZIE-3679) Correct maximum wait time between database retry attempts property
[
https://issues.apache.org/jira/browse/OOZIE-3679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17641942#comment-17641942
]
Janos Makai commented on OOZIE-3679:
hi [~dionusos] can you please take a moment to review the latest patch?
(SpotBugs issues are not related)
Thanks.
> Correct maximum wait time between database retry attempts property
> --
>
> Key: OOZIE-3679
> URL: https://issues.apache.org/jira/browse/OOZIE-3679
> Project: Oozie
> Issue Type: Bug
>Reporter: Janos Makai
>Assignee: Janos Makai
>Priority: Major
> Attachments: OOZIE-3679-001.patch
>
>
> Based on the oozie-default.xml:
> {code:xml}
>
> oozie.service.JPAService.retry.maximum-wait-time.ms
> 3
>
> Maximum wait time between database retry attempts.
>
>
> {code}
> it looks like we are missing a "{_}retry{_}" string before the maximum wait
> time between database retry attempts static string in Oozie's JPAService:
> {code:java}
> public static final String MAX_WAIT_TIME = CONF_PREFIX +
> "maximum-wait-time.ms";
> {code}
> {color:#172b4d}The goal of this change is to sync the config names between
> oozie-default.xml and Oozie but we should also support the old, invalid
> config name.{color}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (OOZIE-3606) Extend file system EL functions to use custom file system properties
[ https://issues.apache.org/jira/browse/OOZIE-3606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17641847#comment-17641847 ] Janos Makai commented on OOZIE-3606: Hello [~dionusos], could you please review the latest patch? I've checked the SpotBugs results, and the issues are unrelated to this change. > Extend file system EL functions to use custom file system properties > > > Key: OOZIE-3606 > URL: https://issues.apache.org/jira/browse/OOZIE-3606 > Project: Oozie > Issue Type: Improvement > Components: core >Affects Versions: 5.2.0 >Reporter: Dénes Bodó >Assignee: Janos Makai >Priority: Major > Attachments: OOZIE-3606-001.patch, OOZIE-3606-002.patch, > OOZIE-3606-003.patch > > > In case we want to check file availability on abfss file system in scope of a > decision node and we have access to that file system only using a key file > which is password protected then currently we are blocked as we cannot allow > configuring hadoop.security.credstore.java-keystore-provider.password-file > globally. > This is why this ticket is created to propose a solution for this issue: > * allow special file system configurations only for file system EL functions > * let them configure globally via oozie-site.xml and for every workflow > independently -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (OOZIE-3678) Reduce the number of NameNode access when starting the Yarn job
[
https://issues.apache.org/jira/browse/OOZIE-3678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17641845#comment-17641845
]
Janos Makai commented on OOZIE-3678:
Hi [~dionusos],
can you please check the latest patch set? The SpotBugs bugs are unrelated to
this change.
Thank you in advance
> Reduce the number of NameNode access when starting the Yarn job
> ---
>
> Key: OOZIE-3678
> URL: https://issues.apache.org/jira/browse/OOZIE-3678
> Project: Oozie
> Issue Type: Improvement
>Affects Versions: 5.2.1
>Reporter: Janos Makai
>Assignee: Janos Makai
>Priority: Major
> Attachments: OOZIE-3678-001.patch, OOZIE-3678-002.patch
>
>
> Before Oozie starts the Yarn application it assembles the class path which's
> elements' paths are resolved with
> {code:java}
> remoteFS.resolvePath(p){code}
> which can cause slowness and high NameNode load.
> The intention is to make this path resolution configurable (ability to turn
> it on/off) via an Oozie configuration thus customers can turn it off when
> they experience job start slowness. Default value should result the new
> behaviour is active.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (OOZIE-3680) Add default value to custom configuration of all the supported file systems in Oozie
Janos Makai created OOZIE-3680: -- Summary: Add default value to custom configuration of all the supported file systems in Oozie Key: OOZIE-3680 URL: https://issues.apache.org/jira/browse/OOZIE-3680 Project: Oozie Issue Type: Improvement Affects Versions: 5.2.1 Reporter: Janos Makai Assignee: Janos Makai The default supported file systems now include: hdfs, hftp, webhdfs, s3a, abfs, abfss. The goal of this ticket is to make sure the users don't have to set explicitely the oozie.service.HadoopAccessorService.supported.filesystems property if they want to use any of the above mentioned supported file systems. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (OOZIE-3679) Correct maximum wait time between database retry attempts property
[
https://issues.apache.org/jira/browse/OOZIE-3679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Janos Makai updated OOZIE-3679:
---
Description:
Based on the oozie-default.xml:
{code:xml}
oozie.service.JPAService.retry.maximum-wait-time.ms
3
Maximum wait time between database retry attempts.
{code}
it looks like we are missing a "{_}retry{_}" string before the maximum wait
time between database retry attempts static string in Oozie's JPAService:
{code:java}
public static final String MAX_WAIT_TIME = CONF_PREFIX + "maximum-wait-time.ms";
{code}
{color:#172b4d}The goal of this change is to sync the config names between
oozie-default.xml and Oozie but we should also support the old, invalid config
name.{color}
was:
Based on the oozie-default.xml:
{code:xml}
oozie.service.JPAService.retry.maximum-wait-time.ms
3
Maximum wait time between database retry attempts.
{code}
it looks like we are missing a "{_}retry{_}" string before the maximum wait
time between database retry attempts static string in Oozie's JPAService:
{code:java}
public static final String MAX_WAIT_TIME = CONF_PREFIX + "maximum-wait-time.ms";
{code}
{color:#172b4d}The goal this change is to sync the config names between
oozie-default.xml and Oozie but we should also support the old, invalid config
name.{color}
> Correct maximum wait time between database retry attempts property
> --
>
> Key: OOZIE-3679
> URL: https://issues.apache.org/jira/browse/OOZIE-3679
> Project: Oozie
> Issue Type: Bug
>Reporter: Janos Makai
>Assignee: Janos Makai
>Priority: Major
>
> Based on the oozie-default.xml:
> {code:xml}
>
> oozie.service.JPAService.retry.maximum-wait-time.ms
> 3
>
> Maximum wait time between database retry attempts.
>
>
> {code}
> it looks like we are missing a "{_}retry{_}" string before the maximum wait
> time between database retry attempts static string in Oozie's JPAService:
> {code:java}
> public static final String MAX_WAIT_TIME = CONF_PREFIX +
> "maximum-wait-time.ms";
> {code}
> {color:#172b4d}The goal of this change is to sync the config names between
> oozie-default.xml and Oozie but we should also support the old, invalid
> config name.{color}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (OOZIE-3679) Correct maximum wait time between database retry attempts property
Janos Makai created OOZIE-3679:
--
Summary: Correct maximum wait time between database retry attempts
property
Key: OOZIE-3679
URL: https://issues.apache.org/jira/browse/OOZIE-3679
Project: Oozie
Issue Type: Bug
Reporter: Janos Makai
Assignee: Janos Makai
Based on the oozie-default.xml:
{code:xml}
oozie.service.JPAService.retry.maximum-wait-time.ms
3
Maximum wait time between database retry attempts.
{code}
it looks like we are missing a "{_}retry{_}" string before the maximum wait
time between database retry attempts static string in Oozie's JPAService:
{code:java}
public static final String MAX_WAIT_TIME = CONF_PREFIX + "maximum-wait-time.ms";
{code}
{color:#172b4d}The goal this change is to sync the config names between
oozie-default.xml and Oozie but we should also support the old, invalid config
name.{color}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (OOZIE-3678) Reduce the number of NameNode access when starting the Yarn job
[
https://issues.apache.org/jira/browse/OOZIE-3678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Janos Makai updated OOZIE-3678:
---
Attachment: OOZIE-3678-002.patch
> Reduce the number of NameNode access when starting the Yarn job
> ---
>
> Key: OOZIE-3678
> URL: https://issues.apache.org/jira/browse/OOZIE-3678
> Project: Oozie
> Issue Type: Improvement
>Affects Versions: 5.2.1
>Reporter: Janos Makai
>Assignee: Janos Makai
>Priority: Major
> Attachments: OOZIE-3678-001.patch, OOZIE-3678-002.patch
>
>
> Before Oozie starts the Yarn application it assembles the class path which's
> elements' paths are resolved with
> {code:java}
> remoteFS.resolvePath(p){code}
> which can cause slowness and high NameNode load.
> The intention is to make this path resolution configurable (ability to turn
> it on/off) via an Oozie configuration thus customers can turn it off when
> they experience job start slowness. Default value should result the new
> behaviour is active.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (OOZIE-3606) Extend file system EL functions to use custom file system properties
[ https://issues.apache.org/jira/browse/OOZIE-3606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Janos Makai updated OOZIE-3606: --- Attachment: OOZIE-3606-003.patch > Extend file system EL functions to use custom file system properties > > > Key: OOZIE-3606 > URL: https://issues.apache.org/jira/browse/OOZIE-3606 > Project: Oozie > Issue Type: Improvement > Components: core >Affects Versions: 5.2.0 >Reporter: Dénes Bodó >Assignee: Janos Makai >Priority: Major > Attachments: OOZIE-3606-001.patch, OOZIE-3606-002.patch, > OOZIE-3606-003.patch > > > In case we want to check file availability on abfss file system in scope of a > decision node and we have access to that file system only using a key file > which is password protected then currently we are blocked as we cannot allow > configuring hadoop.security.credstore.java-keystore-provider.password-file > globally. > This is why this ticket is created to propose a solution for this issue: > * allow special file system configurations only for file system EL functions > * let them configure globally via oozie-site.xml and for every workflow > independently -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (OOZIE-3606) Extend file system EL functions to use custom file system properties
[ https://issues.apache.org/jira/browse/OOZIE-3606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Janos Makai reassigned OOZIE-3606: -- Assignee: Janos Makai (was: Dénes Bodó) > Extend file system EL functions to use custom file system properties > > > Key: OOZIE-3606 > URL: https://issues.apache.org/jira/browse/OOZIE-3606 > Project: Oozie > Issue Type: Improvement > Components: core >Affects Versions: 5.2.0 >Reporter: Dénes Bodó >Assignee: Janos Makai >Priority: Major > Attachments: OOZIE-3606-001.patch, OOZIE-3606-002.patch, > OOZIE-3606-003.patch > > > In case we want to check file availability on abfss file system in scope of a > decision node and we have access to that file system only using a key file > which is password protected then currently we are blocked as we cannot allow > configuring hadoop.security.credstore.java-keystore-provider.password-file > globally. > This is why this ticket is created to propose a solution for this issue: > * allow special file system configurations only for file system EL functions > * let them configure globally via oozie-site.xml and for every workflow > independently -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (OOZIE-3678) Reduce the number of NameNode access when starting the Yarn job
Janos Makai created OOZIE-3678:
--
Summary: Reduce the number of NameNode access when starting the
Yarn job
Key: OOZIE-3678
URL: https://issues.apache.org/jira/browse/OOZIE-3678
Project: Oozie
Issue Type: Improvement
Affects Versions: 5.2.1
Reporter: Janos Makai
Assignee: Janos Makai
Before Oozie starts the Yarn application it assembles the class path which's
elements' paths are resolved with
{code:java}
remoteFS.resolvePath(p){code}
which can cause slowness and high NameNode load.
The intention is to make this path resolution configurable (ability to turn it
on/off) via an Oozie configuration thus customers can turn it off when they
experience job start slowness. Default value should result the new behaviour is
active.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (OOZIE-3677) Oozie should accept a keyStoreType and trustStoreType property in oozie-site.xml
[
https://issues.apache.org/jira/browse/OOZIE-3677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17641326#comment-17641326
]
Janos Makai commented on OOZIE-3677:
Hi [~dionusos] ,
I've uploaded a patch set to this ticket, could you please have a check on
that? :)
> Oozie should accept a keyStoreType and trustStoreType property in
> oozie-site.xml
>
>
> Key: OOZIE-3677
> URL: https://issues.apache.org/jira/browse/OOZIE-3677
> Project: Oozie
> Issue Type: Improvement
>Affects Versions: 5.2.1
>Reporter: Janos Makai
>Assignee: Janos Makai
>Priority: Major
> Attachments: OOZIE-3677-001.patch
>
>
> In scope of making Oozie FIPS compliant, when TLS is enabled, the Oozie
> server expects a JKS KeyStore type even though the following Java system
> properties are being set:
> {code:java}
> -Djavax.net.ssl.keyStoreType=bcfks -Djavax.net.ssl.trustStoreType=bcfks{code}
> The goal of ticket is to make Oozie capable of setting _keyStoreType_ and
> _trustStoreType_ in oozie-site.xml if it's not already set by the user.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (OOZIE-3677) Oozie should accept a keyStoreType and trustStoreType property in oozie-site.xml
[
https://issues.apache.org/jira/browse/OOZIE-3677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Janos Makai updated OOZIE-3677:
---
Description:
In scope of making Oozie FIPS compliant, when TLS is enabled, the Oozie server
expects a JKS KeyStore type even though the following Java system properties
are being set:
{code:java}
-Djavax.net.ssl.keyStoreType=bcfks -Djavax.net.ssl.trustStoreType=bcfks{code}
The goal of ticket is to make Oozie capable of setting _keyStoreType_ and
_trustStoreType_ in oozie-site.xml if it's not already set by the user.
was:The goal of ticket is to make Oozie capable of setting _keyStoreType_ and
_trustStoreType_ in oozie-site.xml if it's not already set by the user.
> Oozie should accept a keyStoreType and trustStoreType property in
> oozie-site.xml
>
>
> Key: OOZIE-3677
> URL: https://issues.apache.org/jira/browse/OOZIE-3677
> Project: Oozie
> Issue Type: Improvement
>Affects Versions: 5.2.1
>Reporter: Janos Makai
>Assignee: Janos Makai
>Priority: Major
> Attachments: OOZIE-3677-001.patch
>
>
> In scope of making Oozie FIPS compliant, when TLS is enabled, the Oozie
> server expects a JKS KeyStore type even though the following Java system
> properties are being set:
> {code:java}
> -Djavax.net.ssl.keyStoreType=bcfks -Djavax.net.ssl.trustStoreType=bcfks{code}
> The goal of ticket is to make Oozie capable of setting _keyStoreType_ and
> _trustStoreType_ in oozie-site.xml if it's not already set by the user.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
