[
https://issues.apache.org/jira/browse/TEZ-988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13955740#comment-13955740
]
Bikas Saha commented on TEZ-988:
--------------------------------
I am sorry, my previous comment was missing the code snippet.
Should this be under some keepAlive check or is this unrelated to keep alive?
{code}+
+ System.setProperty(URL_CONNECTION_ERROR_STREAM_BUFFER_ENABLED,
"true");{code}
Should this just be url.append("&keepAlive=true");
{code}+ if (keepAlive) {
+ url.append("&keepAlive=").append(keepAlive);
+ }{code}
For the interest of others reading this jira, could you please add a short note
on what we are keeping alive and provide some basis for why the default is 5.
E.g. if we are keeping alive connections across 5 different nodes then is the
value too small? Or are we keeping alive 5 different connections to the same
node in which case its too large since we try to fetch everything from a node
in a single URL.
> http.maxConnections needs to be configurable in Tez Fetcher & read from
> errorstream to make the connection reusable
> -------------------------------------------------------------------------------------------------------------------
>
> Key: TEZ-988
> URL: https://issues.apache.org/jira/browse/TEZ-988
> Project: Apache Tez
> Issue Type: Bug
> Affects Versions: 0.4.0
> Reporter: Rajesh Balamohan
> Assignee: Rajesh Balamohan
> Attachments: TEZ-988-v1.patch, TEZ-988-v2.patch, TEZ-988-v3.patch,
> TEZ-988-v4.patch
>
>
> 1. Currently http.maxConnections is set to 5 (default). Make this
> configurable in Fetcher.java. This will help in running larger queries
> 2. ErrorStream has to be read completely in order to make the connection
> reusable (when keepAlive is enabled). Currently, we do not read error stream.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
