[
https://issues.apache.org/jira/browse/HBASE-2387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-2387:
----------------------------------
Description:
FUSE: http://fuse.sourceforge.net/
Create a FUSE translator that mounts exported tablespaces into the Linux
filesystem namespace. Should run in either of two modes:
1) Map the exported tablespace under the mount point.
- If backing with Stargate this is a 1:1 map of resource paths to
filesystem paths.
- Via Thrift or Avro connector, should have the same mapping but
implementation will be more involved.
2) Emulate a filesystem, like s3fs
(http://code.google.com/p/s3fs/wiki/FuseOverAmazon)
- Translate paths under the mount point to row keys for good load
spreading, {{/a/b/c/file.ext}} becomes {{file.ext/c/b/a}}
- Consider borrowing from Tom White's Hadoop S3 FS (HADOOP-574), and store
file data as blocks.
-- After fetching the inode can stream all blocks, e.g. via a multiget
if available. This would support arbitrary file sizes. Otherwise there is a
practical limit somewhere around 20-50 MB with default regionserver heaps.
-- So, {{file.ext/c/b/a}} gets the inode. Blocks would be keyed using
the SHA-1 hash of their contents.
-- Use multiversioning on the inode to get snapshots for free: A path
in the filesystem like {{/a/b/c/file.ext;timestamp}} gets file contents on or
before _timestamp_.
-- Because new writes produce new blocks with unique hashes, this is
like a dedup filesystem. Use ICV to maintain use counters on blocks.
If backing with Stargate, support its multiuser mode.
was:
FUSE: http://fuse.sourceforge.net/
Create a FUSE translator that mounts exported tablespaces into the Linux
filesystem namespace. Should run in either of two modes:
1) Map the exported tablespace under the mount point.
- When using stargate this is a 1:1 map of URL paths to filesystem paths.
- Via Thrift should have the same mapping but implementation will be more
involved.
2) Emulate a filesystem, like s3fs
(http://code.google.com/p/s3fs/wiki/FuseOverAmazon)
- Translate paths under the mount point to row keys for good load
spreading, {{/a/b/c/file.ext}} becomes {{file.ext/c/b/a}}
- Consider borrowing from Tom White's Hadoop S3 FS (HADOOP-574), and store
file data as blocks.
-- After fetching the inode can stream all blocks, e.g. via a Stargate
multiget. This would support arbitrary file sizes. Otherwise there is a
practical limit somewhere around 20-50 MB with default regionserver heaps.
-- So, {{file.ext/c/b/a}} gets the inode. Blocks would be keyed using
the SHA-1 hash of their contents.
-- Use multiversioning on the inode to get snapshots for free: A path
in the filesystem like {{/a/b/c/file.ext;timestamp}} gets file contents on or
before _timestamp_.
-- Because new writes produce new blocks with unique hashes, this is
like a dedup filesystem. Use ICV to maintain use counters on blocks.
- Stargate multiget and multiput operations can help performance. I don't
think Thrift has a comparable multi-op capability.
Support Stargate multiuser mode.
Summary: FUSE module for mounting exported tablespaces (was:
[stargate] FUSE module for mounting exported tablespaces)
> FUSE module for mounting exported tablespaces
> ---------------------------------------------
>
> Key: HBASE-2387
> URL: https://issues.apache.org/jira/browse/HBASE-2387
> Project: Hadoop HBase
> Issue Type: New Feature
> Reporter: Andrew Purtell
> Assignee: Andrew Purtell
> Priority: Minor
>
> FUSE: http://fuse.sourceforge.net/
> Create a FUSE translator that mounts exported tablespaces into the Linux
> filesystem namespace. Should run in either of two modes:
> 1) Map the exported tablespace under the mount point.
> - If backing with Stargate this is a 1:1 map of resource paths to
> filesystem paths.
> - Via Thrift or Avro connector, should have the same mapping but
> implementation will be more involved.
> 2) Emulate a filesystem, like s3fs
> (http://code.google.com/p/s3fs/wiki/FuseOverAmazon)
> - Translate paths under the mount point to row keys for good load
> spreading, {{/a/b/c/file.ext}} becomes {{file.ext/c/b/a}}
> - Consider borrowing from Tom White's Hadoop S3 FS (HADOOP-574), and
> store file data as blocks.
> -- After fetching the inode can stream all blocks, e.g. via a
> multiget if available. This would support arbitrary file sizes. Otherwise
> there is a practical limit somewhere around 20-50 MB with default
> regionserver heaps.
> -- So, {{file.ext/c/b/a}} gets the inode. Blocks would be keyed
> using the SHA-1 hash of their contents.
> -- Use multiversioning on the inode to get snapshots for free: A path
> in the filesystem like {{/a/b/c/file.ext;timestamp}} gets file contents on or
> before _timestamp_.
> -- Because new writes produce new blocks with unique hashes, this is
> like a dedup filesystem. Use ICV to maintain use counters on blocks.
> If backing with Stargate, support its multiuser mode.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
