[
https://issues.apache.org/jira/browse/HDFS-12217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Manoj Govindassamy updated HDFS-12217:
--------------------------------------
Attachment: HDFS-12217.02.patch
Thanks for the detailed review [~yzhangal]. Attached v02 patch with following
addressed. Please take a look at the latest patch.
bq. 1. we can use package wide scope for the isFileDeleted method instead of
protected
Done.
bq. 2. f can't be null in the code, I mean, either an exception will be thrown,
or f will valid value.
True. But, f.get() can be null if the task didn't get to run properly.
Enclosed this code section in try{}catch{} block and now throwing IOException()
on any errors.
bq. 3. If any exception is thrown in the above code, we can log a waning, and
rethrow the exception, because the exception likely indicate a real problem
that we don't expect to happen. If we continue, we might get inconsistent
snapshot.
Makes sense. Now throwing the exception, {{addSnasphot()}} caller handles the
exception and rethrows it as SnaphotException to the upper layer to abort the
snapshot creation.
> HDFS snapshots doesn't capture all open files when one of the open files is
> deleted
> -----------------------------------------------------------------------------------
>
> Key: HDFS-12217
> URL: https://issues.apache.org/jira/browse/HDFS-12217
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: snapshots
> Affects Versions: 3.0.0-alpha1
> Reporter: Manoj Govindassamy
> Assignee: Manoj Govindassamy
> Attachments: HDFS-12217.01.patch, HDFS-12217.02.patch
>
>
> With the fix for HDFS-11402, HDFS Snapshots can additionally capture all the
> open files. Just like all other files, these open files in the snapshots will
> remain immutable. But, sometimes it is found that snapshots fail to capture
> all the open files in the system.
> Under the following conditions, LeaseManager will fail to find INode
> corresponding to an active lease
> * a file is opened for writing (LeaseManager allots a lease), and
> * the same file is deleted while it is still open for writing and having
> active lease, and
> * the same file is not referenced in any other Snapshots/Trash
> {{INode[] LeaseManager#getINodesWithLease()}} can thus return null for few
> leases there by causing the caller to trip over and not return all the open
> files needed by the snapshot manager.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
