[jira] [Resolved] (HADOOP-12884) A failed atomic folder rename operation can never recovery for the destination file is deleted in Wasb filesystem
[
https://issues.apache.org/jira/browse/HADOOP-12884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Liu Shaohui resolved HADOOP-12884.
--
Resolution: Duplicate
> A failed atomic folder rename operation can never recovery for the
> destination file is deleted in Wasb filesystem
> -
>
> Key: HADOOP-12884
> URL: https://issues.apache.org/jira/browse/HADOOP-12884
> Project: Hadoop Common
> Issue Type: Bug
>Reporter: Liu Shaohui
>Priority: Critical
>
> When using Wsab file system, we found that a failed atomic folder rename
> operation can never recovery for the destination file deleted in Wasb
> filesystem.
> {code}
> ls: Attempting to complete rename of file
> hbase/azurtst-xiaomi/data/default/YCSBTest/.tabledesc during folder rename
> redo, and file was not found in source or destination.
> {code}
> The reason is the the file is renamed to the destination file before the
> crash, and the destination file is deleted by another process after crash. So
> the recovery is blocked during finishing the rename operation of this file
> when found the source and destination files all don't exist.
> See: NativeAzureFileSystem.java #finishSingleFileRename
> Another serious problem is that the recovery of atomic rename operation may
> delete new created file which is same name as the source file, because the
> file system don't check if there are rename operation need be redo.
> Suggestions are welcomed~
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HADOOP-12719) Implement lease recover in wasb file system
Liu Shaohui created HADOOP-12719: Summary: Implement lease recover in wasb file system Key: HADOOP-12719 URL: https://issues.apache.org/jira/browse/HADOOP-12719 Project: Hadoop Common Issue Type: New Feature Reporter: Liu Shaohui HBase depend on the lease recovery of file system to prevent the write operations from the "dead" regionservers. After HADOOP-9629, hadoop supports Azure Storage as an alternative Hadoop Compatible File System. If we want to deploy HBase on Azure storage and keep data safety, we need to implement lease recover in wasb file system. [~cnauroth] [~chuanliu] I don't know much about wasb. Any suggestions about how to implements this? Eg, changing permission of the hlog to readonly like FSMapRUtils does? Thanks~ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HADOOP-9971) Pack hadoop compress native libs and upload it to maven for other projects to depend on
Liu Shaohui created HADOOP-9971: --- Summary: Pack hadoop compress native libs and upload it to maven for other projects to depend on Key: HADOOP-9971 URL: https://issues.apache.org/jira/browse/HADOOP-9971 Project: Hadoop Common Issue Type: Improvement Reporter: Liu Shaohui Priority: Minor Attachments: HADOOP-9971-trunk-v1.diff Currently, if other projects like hbase want to using hadoop common native lib, they must copy the native libs to their distribution, which is not agile. From the idea of hadoop-snappy(http://code.google.com/p/hadoop-snappy), we can pack the hadoop common native lib and upload it to maven repository for other projects to depend on. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
