[ https://issues.apache.org/jira/browse/HDFS-11846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16022171#comment-16022171 ]
Weiwei Yang edited comment on HDFS-11846 at 5/24/17 1:51 AM: ------------------------------------------------------------- Thanks [~xyao] for reviewing this. I uploaded v2 patch which has following changes bq. OzoneClientUtils.java: Do we need a configure for HTTP Connection Timeout? Added another property {{OZONE_CLIENT_CONNECTION_TIMEOUT_MS}} for the connection timeout. bq. OzoneBucket.java: we should ensure the outPutStream is closed properly in the final section as well. I have fixed this line, and I have reviewed all streams in these classes and make sure they are closed in final statement as well. bq. OzoneClient.java: unnecessary change. Fixed bq. OzoneClient.java: httppost-> httpPost? I have fixed all such names in {{OzoneClient}}, {{OzoneVolume}} and {{OzoneBucket}} to make sure they all be consistent. bq. OzoneClient.java: I think we should use PoolingHttpClientConnectionManager instead of creating a new connection for each request. I tried to use PoolingHttpClientConnectionManager but I found it doesn't work in current code base. The problem was on the netty server side. It doesn't serve a reused http connection well, I have created HDFS-11873 for that. Thank you. was (Author: cheersyang): Thanks [~xyao] for reviewing this. I uploaded v2 patch which has following changes bq. OzoneClientUtils.java: Do we need a configure for HTTP Connection Timeout? Added another property {{OZONE_CLIENT_CONNECTION_TIMEOUT_MS}} for the connection timeout. bq. OzoneBucket.java: we should ensure the outPutStream is closed properly in the final section as well. I have fixed this line, and I have reviewed all streams in these classes and make sure they are closed in final statement as well. bq. OzoneClient.java: unnecessary change. Fixed bq. OzoneClient.java: httppost-> httpPost? I have fixed all such names in {{OzoneClient}}, {{OzoneVolume}} and {{OzoneBucket}} to make sure they all be consistent. bq. OzoneClient.java: I think we should use PoolingHttpClientConnectionManager instead of creating a new connection for each request. I tried to use PoolingHttpClientConnectionManager but I found it doesn't work in current code base. The problem was on the netty server side. It doesn't serve a reused http connection well, I think we need a new JIRA to fix that. Thank you. > Ozone: Potential http connection leaks in ozone clients > ------------------------------------------------------- > > Key: HDFS-11846 > URL: https://issues.apache.org/jira/browse/HDFS-11846 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: ozone > Reporter: Weiwei Yang > Assignee: Weiwei Yang > Attachments: HDFS-11846-HDFS-7240.001.patch, > HDFS-11846-HDFS-7240.002.patch > > > There are several problems > # Http clients in {{OzoneVolume}}, {{OzoneBucket}} and {{OzoneClient}} are > created per request, per [Reuse of HttpClient > instance|http://hc.apache.org/httpclient-3.x/performance.html#Reuse_of_HttpClient_instance] > doc, proposed to reuse the http client instance to reduce the over head. > # Some resources in these classes were not properly cleaned up. E.g the http > connection, HttpGet/HttpPost requests. > > This jira's purpose is to fix these issues and investigate how we can improve > the client. -- This message was sent by Atlassian JIRA (v6.3.15#6346) --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org