[
https://issues.apache.org/jira/browse/HDFS-10549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15420626#comment-15420626
]
Xiao Chen commented on HDFS-10549:
----------------------------------
Thanks [~linyiqun] for the patch, idea looks good.
I noticed the {{endFileLease}} is done in
{{DFSClient#closeAllFilesBeingWritten}} instead of {{close}} and {{abort}}, any
specific reasons? I think keeping the logic inside is safer, since the problem
is when an exception is thrown within {{close}}/{{abort}}, the line
{{endFileLease}} isn't executed. I hope we can just make sure it's run in any
cases (basically your proposal in HADOOP-13264, but keep the original thrown
exception and rethrow that after we call {{endFileLease}}).
Also, seems {{DFSClient#endFileLease}} does not actually throw any
IOExceptions. That could simplify things a little here too.
Nit:
In the test, can we rephrase this comment?
{code}
// Construct new dfsClient to get same LeaseRenewer instance to avoid
// that the origin dfsClient added in leaseRenewer again.
{code}
Maybe to: Construct a new dfsClient to get the same LeaseRenewer instance, to
avoid the original client being added to the leaseRenewer again.
> Memory leak if exception happens when closing files being written
> -----------------------------------------------------------------
>
> Key: HDFS-10549
> URL: https://issues.apache.org/jira/browse/HDFS-10549
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs-client
> Affects Versions: 2.7.1
> Reporter: Yiqun Lin
> Assignee: Yiqun Lin
> Attachments: HDFS-10549.001.patch, HDFS-10549.002.patch,
> HDFS-10549.003.patch
>
>
> As HADOOP-13264 memtioned, the code dfsClient.endFileLease(fileId) in
> {{DFSOutputStream}} will not be executed when the IOException happened in
> {{closeImpl()}}.
> {code}
> public void close() throws IOException {
> synchronized (this) {
> try (TraceScope ignored =
> dfsClient.newPathTraceScope("DFSOutputStream#close", src)) {
> closeImpl();
> }
> }
> dfsClient.endFileLease(fileId);
> }
> }
> {code}
> This will cause that the files not be closed in {{DFSClient}} and finally
> lead to the memory leak. In {{DFSStripedOutputStream}}, it existed the same
> problem.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
