[jira] [Created] (YARN-10871) Aborted AM is considered as App Failure when user sets MaxAttempts as 1
Prabhu Joseph created YARN-10871:
Summary: Aborted AM is considered as App Failure when user sets
MaxAttempts as 1
Key: YARN-10871
URL: https://issues.apache.org/jira/browse/YARN-10871
Project: Hadoop YARN
Issue Type: Bug
Components: RM
Affects Versions: 3.3.1
Reporter: Prabhu Joseph
Assignee: Prabhu Joseph
When an AM Container is ABORTED due to Node Decommission, the AppAttempt
failure is not counted. But if user sets number of attempts as 1, then YARN
considers the ABORTED AM as a failure.
{code}
int numberOfFailure = app.getNumFailedAppAttempts();
if (app.maxAppAttempts == 1) {
// If the user explicitly set the attempts to 1 then there are likely
// correctness issues if the AM restarts for any reason.
LOG.info("Max app attempts is 1 for " + app.applicationId
+ ", preventing further attempts.");
numberOfFailure = app.maxAppAttempts;
}
{code}
Livy sets the number of attempts as 1 since it's Rpc Server does not yet
support multiple connections for the same registered app. But in our case AM is
ABORTED before even the AM starts (AM was in ACAUIRED state)
Usually users won't decommission the node where the Container is in RUNNING
state (where the session is established). But the decommission can happen on
nodes where the container is in ACQUIRED or ALLOCATED state.
Will suggest to expose an config where user can decide whether to consider this
as a failure or not.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-10871) Aborted AM is considered as App Failure when user sets MaxAttempts as 1
[
https://issues.apache.org/jira/browse/YARN-10871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Prabhu Joseph reassigned YARN-10871:
Assignee: Srinivas S T (was: Prabhu Joseph)
> Aborted AM is considered as App Failure when user sets MaxAttempts as 1
> ---
>
> Key: YARN-10871
> URL: https://issues.apache.org/jira/browse/YARN-10871
> Project: Hadoop YARN
> Issue Type: Bug
> Components: RM
>Affects Versions: 3.3.1
>Reporter: Prabhu Joseph
>Assignee: Srinivas S T
>Priority: Major
>
> When an AM Container is ABORTED due to Node Decommission, the AppAttempt
> failure is not counted. But if user sets number of attempts as 1, then YARN
> considers the ABORTED AM as a failure.
> {code}
> int numberOfFailure = app.getNumFailedAppAttempts();
> if (app.maxAppAttempts == 1) {
> // If the user explicitly set the attempts to 1 then there are likely
> // correctness issues if the AM restarts for any reason.
> LOG.info("Max app attempts is 1 for " + app.applicationId
> + ", preventing further attempts.");
> numberOfFailure = app.maxAppAttempts;
> }
> {code}
> Livy sets the number of attempts as 1 since it's Rpc Server does not yet
> support multiple connections for the same registered app. But in our case AM
> is ABORTED before even the AM starts (AM was in ACAUIRED state)
> Usually users won't decommission the node where the Container is in RUNNING
> state (where the session is established). But the decommission can happen on
> nodes where the container is in ACQUIRED or ALLOCATED state.
> Will suggest to expose an config where user can decide whether to consider
> this as a failure or not.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-10829) Support getApplications API in FederationClientInterceptor
[ https://issues.apache.org/jira/browse/YARN-10829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bibin Chundatt reassigned YARN-10829: - Assignee: Akshat Bordia > Support getApplications API in FederationClientInterceptor > -- > > Key: YARN-10829 > URL: https://issues.apache.org/jira/browse/YARN-10829 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Akshat Bordia >Assignee: Akshat Bordia >Priority: Major > Labels: pull-request-available > Time Spent: 7h 40m > Remaining Estimate: 0h > > Currently getApplications API is not supported in FederationClientInterceptor > and needs to be implemented in FederationClientInterceptor. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Resolved] (YARN-10829) Support getApplications API in FederationClientInterceptor
[ https://issues.apache.org/jira/browse/YARN-10829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bibin Chundatt resolved YARN-10829. --- Fix Version/s: 3.4.0 Resolution: Fixed Committed to trunk > Support getApplications API in FederationClientInterceptor > -- > > Key: YARN-10829 > URL: https://issues.apache.org/jira/browse/YARN-10829 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Akshat Bordia >Assignee: Akshat Bordia >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0 > > Time Spent: 7h 40m > Remaining Estimate: 0h > > Currently getApplications API is not supported in FederationClientInterceptor > and needs to be implemented in FederationClientInterceptor. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-10872) Replace getPropsWithPrefix calls in AutoCreatedQueueTemplate
Andras Gyori created YARN-10872: --- Summary: Replace getPropsWithPrefix calls in AutoCreatedQueueTemplate Key: YARN-10872 URL: https://issues.apache.org/jira/browse/YARN-10872 Project: Hadoop YARN Issue Type: Improvement Components: capacity scheduler Reporter: Andras Gyori Assignee: Andras Gyori With the introduction of YARN-10838, it is now possible to optimise AutoCreatedQueueTemplate and replace calls of getPropsWithPrefix. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10872) Replace getPropsWithPrefix calls in AutoCreatedQueueTemplate
[ https://issues.apache.org/jira/browse/YARN-10872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andras Gyori updated YARN-10872: Parent: YARN-10795 Issue Type: Sub-task (was: Improvement) > Replace getPropsWithPrefix calls in AutoCreatedQueueTemplate > > > Key: YARN-10872 > URL: https://issues.apache.org/jira/browse/YARN-10872 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacity scheduler >Reporter: Andras Gyori >Assignee: Andras Gyori >Priority: Major > > With the introduction of YARN-10838, it is now possible to optimise > AutoCreatedQueueTemplate and replace calls of getPropsWithPrefix. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10838) Implement an optimised version of Configuration getPropsWithPrefix
[ https://issues.apache.org/jira/browse/YARN-10838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andras Gyori updated YARN-10838: Attachment: YARN-10838.003.patch > Implement an optimised version of Configuration getPropsWithPrefix > -- > > Key: YARN-10838 > URL: https://issues.apache.org/jira/browse/YARN-10838 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Andras Gyori >Assignee: Andras Gyori >Priority: Major > Attachments: YARN-10838.001.patch, YARN-10838.002.patch > > > AutoCreatedQueueTemplate also has multiple call to > Configuration#getPropsWithPrefix. It must be eliminated in order to improve > the performance on reinitialisation. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10838) Implement an optimised version of Configuration getPropsWithPrefix
[ https://issues.apache.org/jira/browse/YARN-10838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andras Gyori updated YARN-10838: Attachment: (was: YARN-10838.003.patch) > Implement an optimised version of Configuration getPropsWithPrefix > -- > > Key: YARN-10838 > URL: https://issues.apache.org/jira/browse/YARN-10838 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Andras Gyori >Assignee: Andras Gyori >Priority: Major > Attachments: YARN-10838.001.patch, YARN-10838.002.patch > > > AutoCreatedQueueTemplate also has multiple call to > Configuration#getPropsWithPrefix. It must be eliminated in order to improve > the performance on reinitialisation. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10838) Implement an optimised version of Configuration getPropsWithPrefix
[ https://issues.apache.org/jira/browse/YARN-10838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andras Gyori updated YARN-10838: Attachment: YARN-10838.003.patch > Implement an optimised version of Configuration getPropsWithPrefix > -- > > Key: YARN-10838 > URL: https://issues.apache.org/jira/browse/YARN-10838 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Andras Gyori >Assignee: Andras Gyori >Priority: Major > Attachments: YARN-10838.001.patch, YARN-10838.002.patch, > YARN-10838.003.patch > > > AutoCreatedQueueTemplate also has multiple call to > Configuration#getPropsWithPrefix. It must be eliminated in order to improve > the performance on reinitialisation. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10833) RM logs endpoint vulnerable to clickjacking
[
https://issues.apache.org/jira/browse/YARN-10833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17386268#comment-17386268
]
Benjamin Teke commented on YARN-10833:
--
Thanks for the review [~zhuqi]. Szilard took a look at it before, we were just
waiting for the CI issues to be fixed. Thanks for the help in committing this.
> RM logs endpoint vulnerable to clickjacking
> ---
>
> Key: YARN-10833
> URL: https://issues.apache.org/jira/browse/YARN-10833
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Benjamin Teke
>Assignee: Benjamin Teke
>Priority: Major
> Labels: pull-request-available
> Attachments: YARN-10833.001.patch, YARN-10833.002.patch
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> The /logs endpoint is missing the X-FRAME-OPTIONS in the response header,
> even though YARN is configured to do include it. This makes it vulnerable to
> clickjacking.
> {code:java}
> Request URL: http://{{rm_host}}:8088/logs/
> Request Method: GET
> Status Code: 200 OK
> Remote Address: [::1]:8088
> Referrer Policy: strict-origin-when-cross-origin
> HTTP/1.1 200 OK
> Date: Fri, 25 Jun 2021 17:38:38 GMT
> Cache-Control: no-cache
> Expires: Fri, 25 Jun 2021 17:38:38 GMT
> Date: Fri, 25 Jun 2021 17:38:38 GMT
> Pragma: no-cache
> Content-Type: text/html;charset=utf-8
> X-Content-Type-Options: nosniff
> X-XSS-Protection: 1; mode=block
> Content-Length: 469
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10838) Implement an optimised version of Configuration getPropsWithPrefix
[
https://issues.apache.org/jira/browse/YARN-10838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17386351#comment-17386351
]
Hadoop QA commented on YARN-10838:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Logfile || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
34s{color} | {color:blue}{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} || ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
0s{color} | {color:green}{color} | {color:green} No case conflicting files
found. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green}{color} | {color:green} The patch does not contain any
@author tags. {color} |
| {color:green}+1{color} | {color:green} {color} | {color:green} 0m 0s{color}
| {color:green}test4tests{color} | {color:green} The patch appears to include 3
new or modified test files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} || ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 20m
32s{color} | {color:green}{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
4s{color} | {color:green}{color} | {color:green} trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
57s{color} | {color:green}{color} | {color:green} trunk passed with JDK Private
Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
49s{color} | {color:green}{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
57s{color} | {color:green}{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
15m 4s{color} | {color:green}{color} | {color:green} branch has no errors when
building and testing our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
47s{color} | {color:green}{color} | {color:green} trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
39s{color} | {color:green}{color} | {color:green} trunk passed with JDK Private
Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 18m
22s{color} | {color:blue}{color} | {color:blue} Both FindBugs and SpotBugs are
enabled, using SpotBugs. {color} |
| {color:green}+1{color} | {color:green} spotbugs {color} | {color:green} 1m
53s{color} | {color:green}{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} || ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
50s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
0s{color} | {color:green}{color} | {color:green} the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
0s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
50s{color} | {color:green}{color} | {color:green} the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
50s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 40s{color} |
{color:orange}https://ci-hadoop.apache.org/job/PreCommit-YARN-Build/1137/artifact/out/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt{color}
| {color:orange}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:
The patch generated 1 new + 74 unchanged - 0 fixed = 75 total (was 74) {color}
|
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
54s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green}{color} | {color:green} The patch has no whitespace
issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
13m 26s{color} | {color:green}{color} | {color:green} patch has no errors when
building and testing our client artifacts. {color} |
| {color:r
[jira] [Commented] (YARN-10838) Implement an optimised version of Configuration getPropsWithPrefix
[ https://issues.apache.org/jira/browse/YARN-10838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17386372#comment-17386372 ] Benjamin Teke commented on YARN-10838: -- Thanks [~gandras] for working on this. The latest patch looks good to me, +1 (non-binding). > Implement an optimised version of Configuration getPropsWithPrefix > -- > > Key: YARN-10838 > URL: https://issues.apache.org/jira/browse/YARN-10838 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Andras Gyori >Assignee: Andras Gyori >Priority: Major > Attachments: YARN-10838.001.patch, YARN-10838.002.patch, > YARN-10838.003.patch > > > AutoCreatedQueueTemplate also has multiple call to > Configuration#getPropsWithPrefix. It must be eliminated in order to improve > the performance on reinitialisation. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10838) Implement an optimised version of Configuration getPropsWithPrefix
[
https://issues.apache.org/jira/browse/YARN-10838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17386454#comment-17386454
]
Hadoop QA commented on YARN-10838:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Logfile || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
45s{color} | {color:blue}{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} || ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
0s{color} | {color:green}{color} | {color:green} No case conflicting files
found. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green}{color} | {color:green} The patch does not contain any
@author tags. {color} |
| {color:green}+1{color} | {color:green} {color} | {color:green} 0m 0s{color}
| {color:green}test4tests{color} | {color:green} The patch appears to include 1
new or modified test files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} || ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 2m
21s{color} | {color:blue}{color} | {color:blue} Maven dependency ordering for
branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 20m
6s{color} | {color:green}{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 22m
28s{color} | {color:green}{color} | {color:green} trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 19m
13s{color} | {color:green}{color} | {color:green} trunk passed with JDK Private
Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 3m
28s{color} | {color:green}{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
54s{color} | {color:green}{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
20m 25s{color} | {color:green}{color} | {color:green} branch has no errors when
building and testing our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
3s{color} | {color:green}{color} | {color:green} trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
31s{color} | {color:green}{color} | {color:green} trunk passed with JDK Private
Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 29m
30s{color} | {color:blue}{color} | {color:blue} Both FindBugs and SpotBugs are
enabled, using SpotBugs. {color} |
| {color:green}+1{color} | {color:green} spotbugs {color} | {color:green} 4m
30s{color} | {color:green}{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} || ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
29s{color} | {color:blue}{color} | {color:blue} Maven dependency ordering for
patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
46s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 21m
52s{color} | {color:green}{color} | {color:green} the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 21m
52s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 19m
10s{color} | {color:green}{color} | {color:green} the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 19m
10s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 3m
39s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
44s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green}{color} | {color:green} The patch has no whitespace
issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
13m 36s{color} | {color:green}{color} | {color:green} patch has no errors when
building and testing our client artifacts. {color} |
|
[jira] [Commented] (YARN-10855) yarn logs cli fails to retrieve logs if any TFile is corrupt or empty
[
https://issues.apache.org/jira/browse/YARN-10855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17386458#comment-17386458
]
Jim Brennan commented on YARN-10855:
Thanks [~zhuqi]!
> yarn logs cli fails to retrieve logs if any TFile is corrupt or empty
> -
>
> Key: YARN-10855
> URL: https://issues.apache.org/jira/browse/YARN-10855
> Project: Hadoop YARN
> Issue Type: Bug
> Components: yarn
>Affects Versions: 3.2.2, 2.10.1, 3.4.0, 3.3.1
>Reporter: Jim Brennan
>Assignee: Jim Brennan
>Priority: Major
> Fix For: 3.4.0
>
> Attachments: YARN-10855.001.patch, YARN-10855.002.patch,
> YARN-10855.003.patch
>
>
> When attempting to retrieve yarn logs via the CLI command, it failed with the
> following stack trace (on branch-2.10):
> {noformat}
> yarn logs -applicationId application_1591017890475_1049740 > logs
> 20/06/05 19:15:50 INFO client.RMProxy: Connecting to ResourceManager
> 20/06/05 19:15:51 INFO client.AHSProxy: Connecting to Application History
> server
> Exception in thread "main" java.io.EOFException: Cannot seek to negative
> offset
> at org.apache.hadoop.hdfs.DFSInputStream.seek(DFSInputStream.java:1701)
> at
> org.apache.hadoop.fs.FSDataInputStream.seek(FSDataInputStream.java:65)
> at org.apache.hadoop.io.file.tfile.BCFile$Reader.(BCFile.java:624)
> at org.apache.hadoop.io.file.tfile.TFile$Reader.(TFile.java:804)
> at
> org.apache.hadoop.yarn.logaggregation.AggregatedLogFormat$LogReader.(AggregatedLogFormat.java:503)
> at
> org.apache.hadoop.yarn.logaggregation.LogCLIHelpers.dumpAllContainersLogs(LogCLIHelpers.java:227)
> at org.apache.hadoop.yarn.client.cli.LogsCLI.run(LogsCLI.java:333)
> at org.apache.hadoop.yarn.client.cli.LogsCLI.main(LogsCLI.java:367)
> {noformat}
> The problem was that there was a zero-length TFile for one of the containers
> in the application aggregated log directory in hdfs. When we removed the
> zero length file, {{yarn logs}} was able to retrieve the logs.
> A corrupt or zero length TFile for one container should not prevent loading
> logs for the rest of the application.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10833) RM logs endpoint vulnerable to clickjacking
[
https://issues.apache.org/jira/browse/YARN-10833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17386636#comment-17386636
]
Qi Zhu commented on YARN-10833:
---
Thanks [~bteke] for contribution and [~gandras] [~snemeth] for review.
Committed the PR to trunk.
> RM logs endpoint vulnerable to clickjacking
> ---
>
> Key: YARN-10833
> URL: https://issues.apache.org/jira/browse/YARN-10833
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Benjamin Teke
>Assignee: Benjamin Teke
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.4.0
>
> Attachments: YARN-10833.001.patch, YARN-10833.002.patch
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> The /logs endpoint is missing the X-FRAME-OPTIONS in the response header,
> even though YARN is configured to do include it. This makes it vulnerable to
> clickjacking.
> {code:java}
> Request URL: http://{{rm_host}}:8088/logs/
> Request Method: GET
> Status Code: 200 OK
> Remote Address: [::1]:8088
> Referrer Policy: strict-origin-when-cross-origin
> HTTP/1.1 200 OK
> Date: Fri, 25 Jun 2021 17:38:38 GMT
> Cache-Control: no-cache
> Expires: Fri, 25 Jun 2021 17:38:38 GMT
> Date: Fri, 25 Jun 2021 17:38:38 GMT
> Pragma: no-cache
> Content-Type: text/html;charset=utf-8
> X-Content-Type-Options: nosniff
> X-XSS-Protection: 1; mode=block
> Content-Length: 469
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
