Re: Resource leak in DataSourceNode?

2020-08-30 Thread Robert Metzger
Hi Mark, from the discussion in the JIRA ticket, it seems that we've found somebody in the community who's going to fix this. I don't think calling close() is necessary in the DataSourceNode. The problem is that the connection should not be established in configure() but in open(). Thanks again f

Re: Resource leak in DataSourceNode?

2020-08-30 Thread Mark Davis
Hi Robert, Thank you for confirming that there is an issue. I do not have a solution for it and would like to hear the committer insights what is wrong there. I think there are actually two issues - the first one is the HBase InputFormat does not close a connection in close(). Another is DataSo

Re: Resource leak in DataSourceNode?

2020-08-27 Thread Robert Metzger
Hi Mark, Thanks a lot for your message and the good investigation! I believe you've found a bug in Flink. I filed an issue for the problem: https://issues.apache.org/jira/browse/FLINK-19064. Would you be interested in opening a pull request to fix this? Otherwise, I'm sure a committer will pick u

Resource leak in DataSourceNode?

2020-08-26 Thread Mark Davis
Hi, I am trying to investigate a problem with non-released resources in my application. I have a stateful application which submits Flink DataSetjobs using code very similar to the code in CliFrontend. I noticed what I am getting a lot of non-closed connections to my data store (HBase in my ca