[
https://issues.apache.org/jira/browse/HADOOP-15327?focusedWorklogId=635123&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-635123
]
ASF GitHub Bot logged work on HADOOP-15327:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 06/Aug/21 12:41
Start Date: 06/Aug/21 12:41
Worklog Time Spent: 10m
Work Description: szilard-nemeth commented on a change in pull request
#3259:
URL: https://github.com/apache/hadoop/pull/3259#discussion_r684203696
##########
File path:
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-shuffle/src/main/java/org/apache/hadoop/mapred/ShuffleHandler.java
##########
@@ -920,31 +1002,50 @@ public void channelOpen(ChannelHandlerContext ctx,
ChannelStateEvent evt)
// fetch failure.
headers.put(RETRY_AFTER_HEADER, String.valueOf(FETCH_RETRY_DELAY));
sendError(ctx, "", TOO_MANY_REQ_STATUS, headers);
- return;
+ } else {
+ super.channelActive(ctx);
+ accepted.add(ctx.channel());
+ LOG.debug("Added channel: {}. Accepted number of connections={}",
+ ctx.channel(), acceptedConnections.get());
}
- accepted.add(evt.getChannel());
}
@Override
- public void messageReceived(ChannelHandlerContext ctx, MessageEvent evt)
+ public void channelInactive(ChannelHandlerContext ctx) throws Exception {
+ LOG.trace("Executing channelInactive");
+ super.channelInactive(ctx);
+ acceptedConnections.decrementAndGet();
+ LOG.debug("New value of Accepted number of connections={}",
+ acceptedConnections.get());
+ }
+
+ @Override
+ public void channelRead(ChannelHandlerContext ctx, Object msg)
throws Exception {
- HttpRequest request = (HttpRequest) evt.getMessage();
- if (request.getMethod() != GET) {
+ LOG.trace("Executing channelRead");
+ HttpRequest request = (HttpRequest) msg;
+ LOG.debug("Received HTTP request: {}", request);
+ if (request.method() != GET) {
sendError(ctx, METHOD_NOT_ALLOWED);
return;
}
// Check whether the shuffle version is compatible
+ String shuffleVersion = ShuffleHeader.DEFAULT_HTTP_HEADER_VERSION;
+ if (request.headers() != null) {
+ shuffleVersion = request.headers()
+ .get(ShuffleHeader.HTTP_HEADER_VERSION);
+ }
+ LOG.debug("Shuffle version: {}", shuffleVersion);
if (!ShuffleHeader.DEFAULT_HTTP_HEADER_NAME.equals(
request.headers() != null ?
request.headers().get(ShuffleHeader.HTTP_HEADER_NAME) : null)
|| !ShuffleHeader.DEFAULT_HTTP_HEADER_VERSION.equals(
request.headers() != null ?
- request.headers()
- .get(ShuffleHeader.HTTP_HEADER_VERSION) : null)) {
+ shuffleVersion : null)) {
sendError(ctx, "Incompatible shuffle request version", BAD_REQUEST);
}
Review comment:
Simplified.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 635123)
Time Spent: 2h 40m (was: 2.5h)
> Upgrade MR ShuffleHandler to use Netty4
> ---------------------------------------
>
> Key: HADOOP-15327
> URL: https://issues.apache.org/jira/browse/HADOOP-15327
> Project: Hadoop Common
> Issue Type: Sub-task
> Reporter: Xiaoyu Yao
> Assignee: Szilard Nemeth
> Priority: Major
> Labels: pull-request-available
> Attachments: HADOOP-15327.001.patch, HADOOP-15327.002.patch,
> HADOOP-15327.003.patch, HADOOP-15327.004.patch, HADOOP-15327.005.patch,
> HADOOP-15327.005.patch,
> getMapOutputInfo_BlockingOperationException_awaitUninterruptibly.log,
> testfailure-testMapFileAccess-emptyresponse.zip,
> testfailure-testReduceFromPartialMem.zip
>
> Time Spent: 2h 40m
> Remaining Estimate: 0h
>
> This way, we can remove the dependencies on the netty3 (jboss.netty)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
