[jira] [Commented] (YARN-8072) RM log is getting flooded with MemoryPlacementConstraintManager info logs
[
https://issues.apache.org/jira/browse/YARN-8072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16414320#comment-16414320
]
Hudson commented on YARN-8072:
--
SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #13879 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/13879/])
YARN-8072. RM log is getting flooded with (wangda: rev
24bc5e0e4d919fddae983fc13d4ba4e4e5cb9533)
* (edit)
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/constraint/MemoryPlacementConstraintManager.java
> RM log is getting flooded with MemoryPlacementConstraintManager info logs
> -
>
> Key: YARN-8072
> URL: https://issues.apache.org/jira/browse/YARN-8072
> Project: Hadoop YARN
> Issue Type: Task
>Reporter: Zian Chen
>Assignee: Zian Chen
>Priority: Critical
> Fix For: 3.2.0, 3.1.1
>
> Attachments: YARN-8072.001.patch
>
>
> {quote}Below logs are printed every few seconds or so in RM log
>
> 2018-03-07 05:31:11,858 INFO constraint.MemoryPlacementConstraintManager
> (MemoryPlacementConstraintManager.java:getConstraint(216)) - Application
> application_1520376039316_0001 is not registered in the Placement Constraint
> Manager.
> 2018-03-07 05:31:11,858 INFO constraint.MemoryPlacementConstraintManager
> (MemoryPlacementConstraintManager.java:getConstraint(216)) - Application
> application_1520376039316_0001 is not registered in the Placement Constraint
> Manager.
> 2018-03-07 05:31:11,858 INFO constraint.MemoryPlacementConstraintManager
> (MemoryPlacementConstraintManager.java:getConstraint(216)) - Application
> application_1520376039316_0001 is not registered in the Placement Constraint
> Manager.
> 2018-03-07 05:31:11,859 INFO constraint.MemoryPlacementConstraintManager
> (MemoryPlacementConstraintManager.java:getConstraint(216)) - Application
> application_1520376039316_0001 is not registered in the Placement Constraint
> Manager.
> 2018-03-07 05:31:11,859 INFO constraint.MemoryPlacementConstraintManager
> (MemoryPlacementConstraintManager.java:getConstraint(216)) - Application
> application_1520376039316_0001 is not registered in the Placement Constraint
> Manager.
> 2018-03-07 05:31:11,859 INFO constraint.MemoryPlacementConstraintManager
> (MemoryPlacementConstraintManager.java:getConstraint(216)) - Application
> application_1520376039316_0001 is not registered in the Placement Constraint
> Ma{quote}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8072) RM log is getting flooded with MemoryPlacementConstraintManager info logs
[
https://issues.apache.org/jira/browse/YARN-8072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16414297#comment-16414297
]
Wangda Tan commented on YARN-8072:
--
Just committed to trunk/branch-3.1.
Thanks [~Zian Chen] for the fix, [~cheersyang] for review and [~gsaha] for
reporting the issue.
> RM log is getting flooded with MemoryPlacementConstraintManager info logs
> -
>
> Key: YARN-8072
> URL: https://issues.apache.org/jira/browse/YARN-8072
> Project: Hadoop YARN
> Issue Type: Task
>Reporter: Zian Chen
>Assignee: Zian Chen
>Priority: Critical
> Fix For: 3.1.1
>
> Attachments: YARN-8072.001.patch
>
>
> {quote}Below logs are printed every few seconds or so in RM log
>
> 2018-03-07 05:31:11,858 INFO constraint.MemoryPlacementConstraintManager
> (MemoryPlacementConstraintManager.java:getConstraint(216)) - Application
> application_1520376039316_0001 is not registered in the Placement Constraint
> Manager.
> 2018-03-07 05:31:11,858 INFO constraint.MemoryPlacementConstraintManager
> (MemoryPlacementConstraintManager.java:getConstraint(216)) - Application
> application_1520376039316_0001 is not registered in the Placement Constraint
> Manager.
> 2018-03-07 05:31:11,858 INFO constraint.MemoryPlacementConstraintManager
> (MemoryPlacementConstraintManager.java:getConstraint(216)) - Application
> application_1520376039316_0001 is not registered in the Placement Constraint
> Manager.
> 2018-03-07 05:31:11,859 INFO constraint.MemoryPlacementConstraintManager
> (MemoryPlacementConstraintManager.java:getConstraint(216)) - Application
> application_1520376039316_0001 is not registered in the Placement Constraint
> Manager.
> 2018-03-07 05:31:11,859 INFO constraint.MemoryPlacementConstraintManager
> (MemoryPlacementConstraintManager.java:getConstraint(216)) - Application
> application_1520376039316_0001 is not registered in the Placement Constraint
> Manager.
> 2018-03-07 05:31:11,859 INFO constraint.MemoryPlacementConstraintManager
> (MemoryPlacementConstraintManager.java:getConstraint(216)) - Application
> application_1520376039316_0001 is not registered in the Placement Constraint
> Ma{quote}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8072) RM log is getting flooded with MemoryPlacementConstraintManager info logs
[
https://issues.apache.org/jira/browse/YARN-8072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16414258#comment-16414258
]
Zian Chen commented on YARN-8072:
-
[~cheersyang] , Thank you for your comments Weiwei.
Hi [~leftnoteasy] . could you help me commit this patch? Thanks!
> RM log is getting flooded with MemoryPlacementConstraintManager info logs
> -
>
> Key: YARN-8072
> URL: https://issues.apache.org/jira/browse/YARN-8072
> Project: Hadoop YARN
> Issue Type: Task
>Reporter: Zian Chen
>Assignee: Zian Chen
>Priority: Critical
> Attachments: YARN-8072.001.patch
>
>
> {quote}Below logs are printed every few seconds or so in RM log
>
> 2018-03-07 05:31:11,858 INFO constraint.MemoryPlacementConstraintManager
> (MemoryPlacementConstraintManager.java:getConstraint(216)) - Application
> application_1520376039316_0001 is not registered in the Placement Constraint
> Manager.
> 2018-03-07 05:31:11,858 INFO constraint.MemoryPlacementConstraintManager
> (MemoryPlacementConstraintManager.java:getConstraint(216)) - Application
> application_1520376039316_0001 is not registered in the Placement Constraint
> Manager.
> 2018-03-07 05:31:11,858 INFO constraint.MemoryPlacementConstraintManager
> (MemoryPlacementConstraintManager.java:getConstraint(216)) - Application
> application_1520376039316_0001 is not registered in the Placement Constraint
> Manager.
> 2018-03-07 05:31:11,859 INFO constraint.MemoryPlacementConstraintManager
> (MemoryPlacementConstraintManager.java:getConstraint(216)) - Application
> application_1520376039316_0001 is not registered in the Placement Constraint
> Manager.
> 2018-03-07 05:31:11,859 INFO constraint.MemoryPlacementConstraintManager
> (MemoryPlacementConstraintManager.java:getConstraint(216)) - Application
> application_1520376039316_0001 is not registered in the Placement Constraint
> Manager.
> 2018-03-07 05:31:11,859 INFO constraint.MemoryPlacementConstraintManager
> (MemoryPlacementConstraintManager.java:getConstraint(216)) - Application
> application_1520376039316_0001 is not registered in the Placement Constraint
> Ma{quote}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8072) RM log is getting flooded with MemoryPlacementConstraintManager info logs
[
https://issues.apache.org/jira/browse/YARN-8072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16413269#comment-16413269
]
Weiwei Yang commented on YARN-8072:
---
+1, thanks [~Zian Chen].
> RM log is getting flooded with MemoryPlacementConstraintManager info logs
> -
>
> Key: YARN-8072
> URL: https://issues.apache.org/jira/browse/YARN-8072
> Project: Hadoop YARN
> Issue Type: Task
>Reporter: Zian Chen
>Assignee: Zian Chen
>Priority: Critical
> Attachments: YARN-8072.001.patch
>
>
> {quote}Below logs are printed every few seconds or so in RM log
>
> 2018-03-07 05:31:11,858 INFO constraint.MemoryPlacementConstraintManager
> (MemoryPlacementConstraintManager.java:getConstraint(216)) - Application
> application_1520376039316_0001 is not registered in the Placement Constraint
> Manager.
> 2018-03-07 05:31:11,858 INFO constraint.MemoryPlacementConstraintManager
> (MemoryPlacementConstraintManager.java:getConstraint(216)) - Application
> application_1520376039316_0001 is not registered in the Placement Constraint
> Manager.
> 2018-03-07 05:31:11,858 INFO constraint.MemoryPlacementConstraintManager
> (MemoryPlacementConstraintManager.java:getConstraint(216)) - Application
> application_1520376039316_0001 is not registered in the Placement Constraint
> Manager.
> 2018-03-07 05:31:11,859 INFO constraint.MemoryPlacementConstraintManager
> (MemoryPlacementConstraintManager.java:getConstraint(216)) - Application
> application_1520376039316_0001 is not registered in the Placement Constraint
> Manager.
> 2018-03-07 05:31:11,859 INFO constraint.MemoryPlacementConstraintManager
> (MemoryPlacementConstraintManager.java:getConstraint(216)) - Application
> application_1520376039316_0001 is not registered in the Placement Constraint
> Manager.
> 2018-03-07 05:31:11,859 INFO constraint.MemoryPlacementConstraintManager
> (MemoryPlacementConstraintManager.java:getConstraint(216)) - Application
> application_1520376039316_0001 is not registered in the Placement Constraint
> Ma{quote}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8072) RM log is getting flooded with MemoryPlacementConstraintManager info logs
[
https://issues.apache.org/jira/browse/YARN-8072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16412367#comment-16412367
]
genericqa commented on YARN-8072:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 10m
9s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 27m
24s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
36s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
29s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
38s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
9m 59s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
4s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
29s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
9m 36s{color} | {color:green} patch has no errors when building and testing our
client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
22s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 61m
9s{color} | {color:green} hadoop-yarn-server-resourcemanager in the patch
passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
19s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}125m 13s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:8620d2b |
| JIRA Issue | YARN-8072 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12916012/YARN-8072.001.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient findbugs checkstyle |
| uname | Linux 8fee82cd87eb 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 647058e |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_151 |
| findbugs | v3.1.0-RC1 |
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/20070/testReport/ |
| Max. process+thread count | 848 (vs. ulimit of 1) |
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
|
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/20070/console |
| Powered by | Apache Yetus 0.8.0-SNAPSHOT
[jira] [Commented] (YARN-8072) RM log is getting flooded with MemoryPlacementConstraintManager info logs
[
https://issues.apache.org/jira/browse/YARN-8072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16412322#comment-16412322
]
Wangda Tan commented on YARN-8072:
--
Thanks [~Zian Chen], +1, pending Jenkins.
> RM log is getting flooded with MemoryPlacementConstraintManager info logs
> -
>
> Key: YARN-8072
> URL: https://issues.apache.org/jira/browse/YARN-8072
> Project: Hadoop YARN
> Issue Type: Task
>Reporter: Zian Chen
>Assignee: Zian Chen
>Priority: Major
> Attachments: YARN-8072.001.patch
>
>
> {quote}Below logs are printed every few seconds or so in RM log
>
> 2018-03-07 05:31:11,858 INFO constraint.MemoryPlacementConstraintManager
> (MemoryPlacementConstraintManager.java:getConstraint(216)) - Application
> application_1520376039316_0001 is not registered in the Placement Constraint
> Manager.
> 2018-03-07 05:31:11,858 INFO constraint.MemoryPlacementConstraintManager
> (MemoryPlacementConstraintManager.java:getConstraint(216)) - Application
> application_1520376039316_0001 is not registered in the Placement Constraint
> Manager.
> 2018-03-07 05:31:11,858 INFO constraint.MemoryPlacementConstraintManager
> (MemoryPlacementConstraintManager.java:getConstraint(216)) - Application
> application_1520376039316_0001 is not registered in the Placement Constraint
> Manager.
> 2018-03-07 05:31:11,859 INFO constraint.MemoryPlacementConstraintManager
> (MemoryPlacementConstraintManager.java:getConstraint(216)) - Application
> application_1520376039316_0001 is not registered in the Placement Constraint
> Manager.
> 2018-03-07 05:31:11,859 INFO constraint.MemoryPlacementConstraintManager
> (MemoryPlacementConstraintManager.java:getConstraint(216)) - Application
> application_1520376039316_0001 is not registered in the Placement Constraint
> Manager.
> 2018-03-07 05:31:11,859 INFO constraint.MemoryPlacementConstraintManager
> (MemoryPlacementConstraintManager.java:getConstraint(216)) - Application
> application_1520376039316_0001 is not registered in the Placement Constraint
> Ma{quote}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8072) RM log is getting flooded with MemoryPlacementConstraintManager info logs
[
https://issues.apache.org/jira/browse/YARN-8072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16412269#comment-16412269
]
Zian Chen commented on YARN-8072:
-
Hi [~gsaha], [~leftnoteasy] , this issue is basically changing the logging
level from info to debug to avoid logging flood. Could you help review the
patch and give comments? Thanks!
> RM log is getting flooded with MemoryPlacementConstraintManager info logs
> -
>
> Key: YARN-8072
> URL: https://issues.apache.org/jira/browse/YARN-8072
> Project: Hadoop YARN
> Issue Type: Task
>Reporter: Zian Chen
>Assignee: Zian Chen
>Priority: Major
> Attachments: YARN-8072.001.patch
>
>
> {quote}Below logs are printed every few seconds or so in RM log
>
> 2018-03-07 05:31:11,858 INFO constraint.MemoryPlacementConstraintManager
> (MemoryPlacementConstraintManager.java:getConstraint(216)) - Application
> application_1520376039316_0001 is not registered in the Placement Constraint
> Manager.
> 2018-03-07 05:31:11,858 INFO constraint.MemoryPlacementConstraintManager
> (MemoryPlacementConstraintManager.java:getConstraint(216)) - Application
> application_1520376039316_0001 is not registered in the Placement Constraint
> Manager.
> 2018-03-07 05:31:11,858 INFO constraint.MemoryPlacementConstraintManager
> (MemoryPlacementConstraintManager.java:getConstraint(216)) - Application
> application_1520376039316_0001 is not registered in the Placement Constraint
> Manager.
> 2018-03-07 05:31:11,859 INFO constraint.MemoryPlacementConstraintManager
> (MemoryPlacementConstraintManager.java:getConstraint(216)) - Application
> application_1520376039316_0001 is not registered in the Placement Constraint
> Manager.
> 2018-03-07 05:31:11,859 INFO constraint.MemoryPlacementConstraintManager
> (MemoryPlacementConstraintManager.java:getConstraint(216)) - Application
> application_1520376039316_0001 is not registered in the Placement Constraint
> Manager.
> 2018-03-07 05:31:11,859 INFO constraint.MemoryPlacementConstraintManager
> (MemoryPlacementConstraintManager.java:getConstraint(216)) - Application
> application_1520376039316_0001 is not registered in the Placement Constraint
> Ma{quote}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
