I think it makes sense to move to HDFS if it is reliable (can survive continuous ingest and the agitator) and performs well. Also, I am very curious about what the performance differences are. It would be nice to do some test.
On Mon, Jan 30, 2012 at 10:48 AM, Aaron Cordova <[email protected]> wrote: > At the Hbase vs Accumulo deathmatch the other night Todd elucidated that > Hbase's write-ahead log is in HDFS and benefits somewhat thereby. He > neglected to mention that for years until HDFS append() was available Hbase > just LOST data while Accumulo didn't .. but he was talking about the current > state of affairs so, whatever. > > The question now is, does it make any sense to look at HDFS as a place to > store Accumulo's write-ahead log? I remember that BigTable used two write > streams (each of which is transparently replicated by HDFS) and switched > between them to avoid performance hiccups, so it does sound like a critical > part of the overall performance. Such a big change would belong probably in > 1.6 or later ... But there may be reasons to never use HDFS and to always use > a separately maintained subsystem. > > Any one care to lay out the arguments for staying with a separate subsystem? > I think we know the arguments for using HDFS. > > Aaron >
