[
https://issues.apache.org/jira/browse/HBASE-17437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15840496#comment-15840496
]
stack commented on HBASE-17437:
-------------------------------
[~zyork] [~enis] is referring to the fs redo project. Its lead-in JIRA is
HBASE-14090. The project is currently on-hold labelled as a possible for hbase
2.0 (though unlikely). The idea is that hbase would introduce a logical fs tier
undoing ourselves of a direct mapping to a physical layout and taking on
responsibility for atomic actions (making a file visible, deletes, etc.)
ourselves rather than depend on the fs to provide it to us. Our expectation of
the fs would plummet; in essence we'd treat it as a bucket of files; up in
hbase is where we'd group by store/region/table; not in the fs as is done now.
The redo would be an opportunity to clean up filesystem usage in hbase -- it
has leaked all over the codebase -- and it would allow us scale (currently 1M
regions is untenable given it means there'll be directory in HDFS with 1M
region dirs...) as well as more easily entertain our slotting in another fs
chassis other than HDFS when the Interface has been radically pruned back.
Let me look at this patch up in RB. It was good of @enis to do the fs redo
shout out but currently it doesn't look like it is landing soon so we could
take this on in the meantime. Let me take a look...
> Support specifying a WAL directory outside of the root directory
> ----------------------------------------------------------------
>
> Key: HBASE-17437
> URL: https://issues.apache.org/jira/browse/HBASE-17437
> Project: HBase
> Issue Type: Improvement
> Components: Filesystem Integration, wal
> Affects Versions: 1.2.4
> Reporter: Yishan Yang
> Assignee: Zach York
> Labels: patch
> Fix For: 2.0.0, 1.4.0
>
> Attachments: hbase-17437-branch-1.2.patch,
> HBASE-17437.master.001.patch, HBASE-17437.master.002.patch,
> HBASE-17437.master.003.patch, HBASE-17437.master.004.patch,
> hbase-17437-master.patch
>
>
> Currently, the WAL and the StoreFiles need to be on the same FileSystem. Some
> FileSystems (such as Amazon S3) don’t support append or consistent writes.
> These two properties are imperative for the WAL in order to avoid loss of
> writes. However, StoreFiles don’t necessarily need the same consistency
> guarantees (since writes are cached locally and if writes fail, they can
> always be replayed from the WAL).
>
> This JIRA aims to allow users to configure a log directory (for WALs) that is
> outside of the root directory or even in a different FileSystem. The default
> value will still put the log directory under the root directory.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
