Jack Yang created YARN-11845:
--------------------------------
Summary: CLOSE_WAIT leaks in the WebAppProxyServlet
Key: YARN-11845
URL: https://issues.apache.org/jira/browse/YARN-11845
Project: Hadoop YARN
Issue Type: Bug
Components: webproxy
Affects Versions: 3.3.6
Reporter: Jack Yang
Attachments: image-2025-08-07-16-16-52-193.png
In certain scenarios, we make API calls such as
http://\{rm_address}/proxy/\{appid}/ws/v1/mapreduce/jobs to monitor the state
of running applications. During this process, we noticed a substantial number
of TCP connections accumulating in the CLOSE_WAIT state.
!image-2025-08-07-16-16-52-193.png!
The key reason we think is that the connection is not closed in proxyLink() of
WebAppProxyServlet. The request link is: client -> rm -> appMaster, the rm
would proxy the request to appMaster, when the request done, rm keeps the
connection open but the appMaster side close the connection (FIN sent).
{code:java}
try {
HttpResponse httpResp = client.execute(base);
resp.setStatus(httpResp.getStatusLine().getStatusCode());
for (Header header : httpResp.getAllHeaders()) {
resp.setHeader(header.getName(), header.getValue());
}
if (c != null) {
resp.addCookie(c);
}
InputStream in = httpResp.getEntity().getContent();
if (in != null) {
IOUtils.copyBytes(in, out, 4096, true);
}
} finally {
// the connection is not closed properly
base.releaseConnection();
}{code}
We think there is no need to keep the connection open with upstream in this
scenarios. So maybe add Connection:close header is good choice.
{code:java}
base.setHeader("Connection", "close");{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
