hbase relies on .tmp directories to do some sort of "atomic" file creation. and avoid problems like half data written when it crashes.
there is a jira open, to solve that problem in one of the next major releases: https://issues.apache.org/jira/browse/HBASE-14090 There is a document in it, if you are interested in reading about the internals. Matteo On Wed, Sep 9, 2015 at 4:23 PM, Anthony Nguyen <anthony.an.ngu...@gmail.com> wrote: > Hi all, > > I'm investigating the use of S3 as a backing store for HBase. Would there > be any major issues with modifying HBase in such a way where when an S3 > location is set for the rootdir, writes to .tmp are removed and minimized, > instead writing directly to the final destination? The reason I'd like to > do this is because renames in S3 are expensive and performance for > operations such as compactions and snapshot restores that have many renames > suffer. > > Thanks! >
