[
https://issues.apache.org/jira/browse/MAPREDUCE-7369?focusedWorklogId=766458&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-766458
]
ASF GitHub Bot logged work on MAPREDUCE-7369:
---------------------------------------------
Author: ASF GitHub Bot
Created on: 05/May/22 06:57
Start Date: 05/May/22 06:57
Worklog Time Spent: 10m
Work Description: iwasakims commented on code in PR #4247:
URL: https://github.com/apache/hadoop/pull/4247#discussion_r865603104
##########
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml:
##########
@@ -286,6 +286,13 @@
</description>
</property>
+<property>
+ <name>mapreduce.task.enable.ping-for-liveliness-check</name>
+ <value>true</value>
Review Comment:
When should we turn this off? If it should be always true, we don't need to
make it configurable. We usually needs new configuration knob to add new
feature disabled by default for compatibility and safety.
Issue Time Tracking
-------------------
Worklog Id: (was: 766458)
Time Spent: 1h 10m (was: 1h)
> MapReduce tasks timing out when spends more time on MultipleOutputs#close
> -------------------------------------------------------------------------
>
> Key: MAPREDUCE-7369
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7369
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 3.3.1
> Reporter: Prabhu Joseph
> Assignee: Ashutosh Gupta
> Priority: Major
> Labels: pull-request-available
> Attachments: MAPREDUCE-7369.001.patch
>
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> MapReduce tasks timing out when spends more time on MultipleOutputs#close.
> MultipleOutputs#closes takes more time when there are multiple files to be
> closed & there is a high latency in closing a stream.
> {code}
> 2021-11-01 02:45:08,312 INFO [AsyncDispatcher event handler]
> org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: Diagnostics
> report from attempt_1634949471086_61268_m_001115_0:
> AttemptID:attempt_1634949471086_61268_m_001115_0 Timed out after 300 secs
> {code}
> MapReduce task timeout can be increased but it is tough to set the right
> timeout value. The timeout can be disabled with 0 but that might lead to
> hanging tasks not getting killed.
> The tasks are sending the ping every 3 seconds which are not honored by
> ApplicationMaster. It expects the status information which won't be send
> during MultipleOutputs#close. This jira is to add a config which considers
> the ping from task as part of Task Liveliness Check in the ApplicationMaster.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
