[
https://issues.apache.org/jira/browse/GOBBLIN-1616?focusedWorklogId=748752&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-748752
]
ASF GitHub Bot logged work on GOBBLIN-1616:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 28/Mar/22 16:33
Start Date: 28/Mar/22 16:33
Worklog Time Spent: 10m
Work Description: ZihanLi58 commented on a change in pull request #3486:
URL: https://github.com/apache/gobblin/pull/3486#discussion_r836626979
##########
File path:
gobblin-core/src/main/java/org/apache/gobblin/source/extractor/extract/restapi/RestApiConnector.java
##########
@@ -84,6 +85,10 @@ public RestApiConnector(State state) {
public void close() throws IOException {
// This is to close any idle connections opening by the httpClient
this.closer.close();
+ if (this.getHttpClient() != null && !(this.getHttpClient() instanceof
Closeable)) {
+ log.warn("httpClient is not closable, we will only close the idle
connections");
+ this.getHttpClient().getConnectionManager().closeIdleConnections(0,
TimeUnit.MILLISECONDS);
Review comment:
@phet If it's not a closable http client, we should rely on the
httpclient implementation to close the connection instead of we wrapping them
up in closer API? Without the implementation detail, I don't think we can make
sure to close all the connections, i.e. even we have the response API, I don't
think we have a way to access the underlying socket it's using and close it in
non-closable httpClient. Adding this logic to close idle connection here is
just to try our best to close resources ASAP instead of relying on GC, but it's
not possible/reasonable for us to avoid the resources leak with un-closable
httpClient.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 748752)
Time Spent: 3h 40m (was: 3.5h)
> Make RestApiConnector be able to close the connection finally
> -------------------------------------------------------------
>
> Key: GOBBLIN-1616
> URL: https://issues.apache.org/jira/browse/GOBBLIN-1616
> Project: Apache Gobblin
> Issue Type: Bug
> Reporter: Zihan Li
> Priority: Major
> Time Spent: 3h 40m
> Remaining Estimate: 0h
>
> In RestApiConnector, we only consume the HttpEntity but not close the
> response, which potentially leave the socket as CLOSE_WAIT status and eat up
> the resources.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
