[jira] [Updated] (YARN-2046) Out of band heartbeats are sent only on container kill and possibly too early
[ https://issues.apache.org/jira/browse/YARN-2046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated YARN-2046: - Fix Version/s: 2.8.0 > Out of band heartbeats are sent only on container kill and possibly too early > - > > Key: YARN-2046 > URL: https://issues.apache.org/jira/browse/YARN-2046 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Affects Versions: 0.23.10, 2.4.0 >Reporter: Jason Lowe >Assignee: Ming Ma > Fix For: 2.8.0, 2.7.3, 2.6.5, 3.0.0-alpha1 > > Attachments: YARN-2046-2.patch, YARN-2046-3.patch, YARN-2046-4.patch, > YARN-2046-5.patch, YARN-2046-branch-2.6.patch, YARN-2046-branch-2.7.patch, > YARN-2046.patch > > > [~mingma] pointed out in the review discussion for MAPREDUCE-5465 that the NM > is currently sending out of band heartbeats only when stopContainer is > called. In addition those heartbeats might be sent too early because the > container kill event is asynchronously posted then the heartbeat monitor is > notified. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-2046) Out of band heartbeats are sent only on container kill and possibly too early
[ https://issues.apache.org/jira/browse/YARN-2046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated YARN-2046: - Labels: (was: BB2015-05-RFC) > Out of band heartbeats are sent only on container kill and possibly too early > - > > Key: YARN-2046 > URL: https://issues.apache.org/jira/browse/YARN-2046 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Affects Versions: 0.23.10, 2.4.0 >Reporter: Jason Lowe >Assignee: Ming Ma > Attachments: YARN-2046-2.patch, YARN-2046-3.patch, YARN-2046-4.patch, > YARN-2046-5.patch, YARN-2046-branch-2.6.patch, YARN-2046-branch-2.7.patch, > YARN-2046.patch > > > [~mingma] pointed out in the review discussion for MAPREDUCE-5465 that the NM > is currently sending out of band heartbeats only when stopContainer is > called. In addition those heartbeats might be sent too early because the > container kill event is asynchronously posted then the heartbeat monitor is > notified. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-2046) Out of band heartbeats are sent only on container kill and possibly too early
[ https://issues.apache.org/jira/browse/YARN-2046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ming Ma updated YARN-2046: -- Attachment: YARN-2046-branch-2.6.patch YARN-2046-branch-2.7.patch Thanks [~jlowe]. Agree it is useful to for branch-2.7 and branch 2.6. Here are the specific patches for those two branches. > Out of band heartbeats are sent only on container kill and possibly too early > - > > Key: YARN-2046 > URL: https://issues.apache.org/jira/browse/YARN-2046 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Affects Versions: 0.23.10, 2.4.0 >Reporter: Jason Lowe >Assignee: Ming Ma > Labels: BB2015-05-RFC > Attachments: YARN-2046-2.patch, YARN-2046-3.patch, YARN-2046-4.patch, > YARN-2046-5.patch, YARN-2046-branch-2.6.patch, YARN-2046-branch-2.7.patch, > YARN-2046.patch > > > [~mingma] pointed out in the review discussion for MAPREDUCE-5465 that the NM > is currently sending out of band heartbeats only when stopContainer is > called. In addition those heartbeats might be sent too early because the > container kill event is asynchronously posted then the heartbeat monitor is > notified. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-2046) Out of band heartbeats are sent only on container kill and possibly too early
[ https://issues.apache.org/jira/browse/YARN-2046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ming Ma updated YARN-2046: -- Attachment: YARN-2046-5.patch Thanks [~jlowe]! Here is the updated patch with your suggestion. > Out of band heartbeats are sent only on container kill and possibly too early > - > > Key: YARN-2046 > URL: https://issues.apache.org/jira/browse/YARN-2046 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Affects Versions: 0.23.10, 2.4.0 >Reporter: Jason Lowe >Assignee: Ming Ma > Labels: BB2015-05-RFC > Attachments: YARN-2046-2.patch, YARN-2046-3.patch, YARN-2046-4.patch, > YARN-2046-5.patch, YARN-2046.patch > > > [~mingma] pointed out in the review discussion for MAPREDUCE-5465 that the NM > is currently sending out of band heartbeats only when stopContainer is > called. In addition those heartbeats might be sent too early because the > container kill event is asynchronously posted then the heartbeat monitor is > notified. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-2046) Out of band heartbeats are sent only on container kill and possibly too early
[ https://issues.apache.org/jira/browse/YARN-2046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ming Ma updated YARN-2046: -- Attachment: YARN-2046-4.patch The TestLogAggregationService failure is unrelated. It passes locally. The update patch has fixed one of the checkstyle issues. For the rest of checkstyle issues, there is not much to do. If we want to make sure not to pass too many parameters to constructors, we need to use the builder pattern. > Out of band heartbeats are sent only on container kill and possibly too early > - > > Key: YARN-2046 > URL: https://issues.apache.org/jira/browse/YARN-2046 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Affects Versions: 0.23.10, 2.4.0 >Reporter: Jason Lowe >Assignee: Ming Ma > Labels: BB2015-05-RFC > Attachments: YARN-2046-2.patch, YARN-2046-3.patch, YARN-2046-4.patch, > YARN-2046.patch > > > [~mingma] pointed out in the review discussion for MAPREDUCE-5465 that the NM > is currently sending out of band heartbeats only when stopContainer is > called. In addition those heartbeats might be sent too early because the > container kill event is asynchronously posted then the heartbeat monitor is > notified. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-2046) Out of band heartbeats are sent only on container kill and possibly too early
[ https://issues.apache.org/jira/browse/YARN-2046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ming Ma updated YARN-2046: -- Attachment: YARN-2046-3.patch With out-of-band heartbeat we can afford to set larger NM -> RM heartbeat interval. That is useful when you have a large cluster and large NM -> RM heartbeat interval can reduce the load on RM. Here is the rebased patch. > Out of band heartbeats are sent only on container kill and possibly too early > - > > Key: YARN-2046 > URL: https://issues.apache.org/jira/browse/YARN-2046 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Affects Versions: 0.23.10, 2.4.0 >Reporter: Jason Lowe >Assignee: Ming Ma > Labels: BB2015-05-RFC > Attachments: YARN-2046-2.patch, YARN-2046-3.patch, YARN-2046.patch > > > [~mingma] pointed out in the review discussion for MAPREDUCE-5465 that the NM > is currently sending out of band heartbeats only when stopContainer is > called. In addition those heartbeats might be sent too early because the > container kill event is asynchronously posted then the heartbeat monitor is > notified. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-2046) Out of band heartbeats are sent only on container kill and possibly too early
[ https://issues.apache.org/jira/browse/YARN-2046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ming Ma updated YARN-2046: -- Attachment: YARN-2046-2.patch Thanks [~xgong]. Here is the rebased patch. Out of band heartbeats are sent only on container kill and possibly too early - Key: YARN-2046 URL: https://issues.apache.org/jira/browse/YARN-2046 Project: Hadoop YARN Issue Type: Bug Components: nodemanager Affects Versions: 0.23.10, 2.4.0 Reporter: Jason Lowe Assignee: Ming Ma Attachments: YARN-2046-2.patch, YARN-2046.patch [~mingma] pointed out in the review discussion for MAPREDUCE-5465 that the NM is currently sending out of band heartbeats only when stopContainer is called. In addition those heartbeats might be sent too early because the container kill event is asynchronously posted then the heartbeat monitor is notified. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-2046) Out of band heartbeats are sent only on container kill and possibly too early
[ https://issues.apache.org/jira/browse/YARN-2046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ming Ma updated YARN-2046: -- Labels: BB2015-05-RFC (was: ) Out of band heartbeats are sent only on container kill and possibly too early - Key: YARN-2046 URL: https://issues.apache.org/jira/browse/YARN-2046 Project: Hadoop YARN Issue Type: Bug Components: nodemanager Affects Versions: 0.23.10, 2.4.0 Reporter: Jason Lowe Assignee: Ming Ma Labels: BB2015-05-RFC Attachments: YARN-2046-2.patch, YARN-2046.patch [~mingma] pointed out in the review discussion for MAPREDUCE-5465 that the NM is currently sending out of band heartbeats only when stopContainer is called. In addition those heartbeats might be sent too early because the container kill event is asynchronously posted then the heartbeat monitor is notified. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-2046) Out of band heartbeats are sent only on container kill and possibly too early
[ https://issues.apache.org/jira/browse/YARN-2046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ming Ma updated YARN-2046: -- Attachment: (was: YARN-2046.patch) Out of band heartbeats are sent only on container kill and possibly too early - Key: YARN-2046 URL: https://issues.apache.org/jira/browse/YARN-2046 Project: Hadoop YARN Issue Type: Bug Components: nodemanager Affects Versions: 0.23.10, 2.4.0 Reporter: Jason Lowe Assignee: Ming Ma [~mingma] pointed out in the review discussion for MAPREDUCE-5465 that the NM is currently sending out of band heartbeats only when stopContainer is called. In addition those heartbeats might be sent too early because the container kill event is asynchronously posted then the heartbeat monitor is notified. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-2046) Out of band heartbeats are sent only on container kill and possibly too early
[ https://issues.apache.org/jira/browse/YARN-2046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ming Ma updated YARN-2046: -- Attachment: YARN-2046.patch Out of band heartbeats are sent only on container kill and possibly too early - Key: YARN-2046 URL: https://issues.apache.org/jira/browse/YARN-2046 Project: Hadoop YARN Issue Type: Bug Components: nodemanager Affects Versions: 0.23.10, 2.4.0 Reporter: Jason Lowe Assignee: Ming Ma Attachments: YARN-2046.patch [~mingma] pointed out in the review discussion for MAPREDUCE-5465 that the NM is currently sending out of band heartbeats only when stopContainer is called. In addition those heartbeats might be sent too early because the container kill event is asynchronously posted then the heartbeat monitor is notified. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-2046) Out of band heartbeats are sent only on container kill and possibly too early
[ https://issues.apache.org/jira/browse/YARN-2046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ming Ma updated YARN-2046: -- Attachment: YARN-2046.patch In the patch, the container asks NodeStatusUpdater to send out of band heartbeats when it exits. The definition of exit also includes special cases where the container is killed before it is launched. This allows RM to know about completed containers via out of band heartbeats under different scenarios. Out of band heartbeats are sent only on container kill and possibly too early - Key: YARN-2046 URL: https://issues.apache.org/jira/browse/YARN-2046 Project: Hadoop YARN Issue Type: Bug Components: nodemanager Affects Versions: 0.23.10, 2.4.0 Reporter: Jason Lowe Attachments: YARN-2046.patch [~mingma] pointed out in the review discussion for MAPREDUCE-5465 that the NM is currently sending out of band heartbeats only when stopContainer is called. In addition those heartbeats might be sent too early because the container kill event is asynchronously posted then the heartbeat monitor is notified. -- This message was sent by Atlassian JIRA (v6.2#6252)
