[
https://issues.apache.org/jira/browse/HBASE-6233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13408762#comment-13408762
]
stack commented on HBASE-6233:
------------------------------
@Matteo Thanks for taking the time to do the writeup. Helpful. I like how
your symlink work would make it so no work moving up on to hdfs hard links.
I was wondering if you have any concern around creation of all the symlinks on
a table of some decent size taking a good bit of time Matteo? The window
during which the snapshot is being made could be pretty wide. Would that be a
problem?
You ask for ideas and the only one I have is the hackneyed one copied from
bdbje where on compaction, we do not delete files; rather we just rename them
w/ a '.del' ending and leave them in place. On snapshot, we make a manifest of
all files in the table. On restore, we'd read the manifest and look for files
first w/o the .del and then if not found, with the .del. I've not thought it
all through to the extent of your attached pdf -- I can see how it could get
tangled pretty quickly -- but throwing it up there since you were asking.
bq. ...and can be an external tool or an internal thread that scan the snapshot.
Could hitch a ride on the current meta scanner, the one that cleans the parent
regions from .META.
Adding list of files to .META. might make for our being able to do other
fancyness such as the Accumulo fast table copy, etc.
Let me read your doc. some more (and Jesse's work).
> [brainstorm] snapshots: hardlink alternatives
> ---------------------------------------------
>
> Key: HBASE-6233
> URL: https://issues.apache.org/jira/browse/HBASE-6233
> Project: HBase
> Issue Type: Brainstorming
> Reporter: Matteo Bertozzi
> Assignee: Matteo Bertozzi
> Attachments: Restore-Snapshot-Hardlink-alternatives.pdf
>
>
> Discussion ticket around snapshots and hardlink alternatives.
> (See the HDFS-3370 discussion about hardlink and implementation problems)
> (taking for a moment WAL out of the discussion and focusing on hfiles)
> With hardlinks available taking snapshot will be fairly easy:
> * (hfiles are immutable)
> * hardlink to .snapshot/name to take snapshot
> * hardlink from .snapshot/name to restore the snapshot
> * No code change needed (on fs.delete() only one reference is deleted)
> but we don't have hardlinks, what are the alternatives?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
