[
https://issues.apache.org/jira/browse/MAPREDUCE-7536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18096887#comment-18096887
]
ASF GitHub Bot commented on MAPREDUCE-7536:
-------------------------------------------
konstantinb commented on PR #8427:
URL: https://github.com/apache/hadoop/pull/8427#issuecomment-4995686097
Gentle ping for review — this has been open since April and is a small,
self-contained change.
It extends the existing ignorable-error handling already present in
`exceptionCaught` (`ClosedChannelException` / `IGNORABLE_ERROR_MESSAGE`) into
the `operationComplete` path, so routine client disconnects during `sendMap`
log at DEBUG instead of ERROR. No functional or API change: the channel is
still closed and the future short-circuits exactly as before — the only change
is logging, where client-disconnect causes now take a dedicated DEBUG message
instead of the ERROR line, matching the handling already in `exceptionCaught`.
Includes a parameterized test covering both the ignorable cases and the
genuine-error case.
Merged in recent trunk changes yesterday, and Yetus is green.
@pjfanning would you have a few minutes to take a look, given your recent
Netty work in this area? Thanks!
> ShuffleChannelHandler logs ERROR for client disconnects during shuffle
> ----------------------------------------------------------------------
>
> Key: MAPREDUCE-7536
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7536
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Reporter: Konstantin Bereznyakov
> Priority: Major
> Labels: pull-request-available
>
> ShuffleChannelHandler.operationComplete() logs at ERROR level when a
> ChannelFuture completes unsuccessfully, even for expected conditions like
> client disconnections. This creates unnecessary noise in shuffle service logs.
> *Current Behavior*
> When a shuffle client disconnects during data transfer, the handler logs:
> ERROR Future is unsuccessful. channel='...' Cause: <ClosedChannelException
> or connection reset>
> These are expected during normal operation when reducers timeout, get
> killed, or disconnect early.
> E[xpected
> Behavior|https://github.com/apache/hadoop/blob/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-shuffle/src/main/java/org/apache/hadoop/mapred/ShuffleChannelHandler.java#L679]
> Expected disconnection scenarios should be logged at DEBUG level:
> - ClosedChannelException
> - Connection reset by peer
> - Other ignorable I/O errors matching existing IGNORABLE_ERROR_MESSAGE
> pattern
> Only unexpected failures should be logged at ERROR level.
> *Impact*
> - Log noise in production NodeManager shuffle service logs
> - Difficult to identify real errors among expected disconnections
> - Unnecessary alerting in monitoring systems that track ERROR log volume
> [Affected
> Component|https://github.com/apache/hadoop/blob/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-shuffle/src/main/java/org/apache/hadoop/mapred/ShuffleChannelHandler.java#L679]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
