[ https://issues.apache.org/jira/browse/HDFS-12217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Manoj Govindassamy updated HDFS-12217: -------------------------------------- Attachment: HDFS-12217.02.patch Thanks for the detailed review [~yzhangal]. Attached v02 patch with following addressed. Please take a look at the latest patch. bq. 1. we can use package wide scope for the isFileDeleted method instead of protected Done. bq. 2. f can't be null in the code, I mean, either an exception will be thrown, or f will valid value. True. But, f.get() can be null if the task didn't get to run properly. Enclosed this code section in try{}catch{} block and now throwing IOException() on any errors. bq. 3. If any exception is thrown in the above code, we can log a waning, and rethrow the exception, because the exception likely indicate a real problem that we don't expect to happen. If we continue, we might get inconsistent snapshot. Makes sense. Now throwing the exception, {{addSnasphot()}} caller handles the exception and rethrows it as SnaphotException to the upper layer to abort the snapshot creation. > HDFS snapshots doesn't capture all open files when one of the open files is > deleted > ----------------------------------------------------------------------------------- > > Key: HDFS-12217 > URL: https://issues.apache.org/jira/browse/HDFS-12217 > Project: Hadoop HDFS > Issue Type: Bug > Components: snapshots > Affects Versions: 3.0.0-alpha1 > Reporter: Manoj Govindassamy > Assignee: Manoj Govindassamy > Attachments: HDFS-12217.01.patch, HDFS-12217.02.patch > > > With the fix for HDFS-11402, HDFS Snapshots can additionally capture all the > open files. Just like all other files, these open files in the snapshots will > remain immutable. But, sometimes it is found that snapshots fail to capture > all the open files in the system. > Under the following conditions, LeaseManager will fail to find INode > corresponding to an active lease > * a file is opened for writing (LeaseManager allots a lease), and > * the same file is deleted while it is still open for writing and having > active lease, and > * the same file is not referenced in any other Snapshots/Trash > {{INode[] LeaseManager#getINodesWithLease()}} can thus return null for few > leases there by causing the caller to trip over and not return all the open > files needed by the snapshot manager. -- This message was sent by Atlassian JIRA (v6.4.14#64029) --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org