> > > Thanks for adding it there -- I really think it is a big headline caveat on > my expectation of "eventual consistency". Other systems out there that > give you eventually consistency on the millisecond level for most cases, > while this initial implementation would has eventual mean 10's of minutes > or even handfuls of minutes behind (with the snapshots flush mechanism)!
> There are a handful of other things in the phase one part of the > implementation section that limit the usefulness of the feature to a > certain kind of constrained hbase user. I'll start another thread for > those. > > Yes, hopefully we will not stop with only phase 1, and continue to implement the more-latent async wal replication and/or wal tailing. However phase 1 will get us to the point of demonstrating that replicated regions works, the client side of execution is manageable, and there is real benefit for read-only or bulk loaded tables plus some specific use cases for read/write tables. > > > > > We are proposing to implement "Region snapshots" first and "Async wal > > replication" second. > > As argued, I think wal-tailing only makes sense with WALpr so, that work > is > > left until after we have WAL > > per region. > > > > > This is our main disagreement -- I'm not convinced that wal tailing only > making sense for the wal per region hlog implementation. Instead of > bouncing around hypotheticals, it sounds like I'll be doing more > experiments to prove it to myself and to convince you. :) > That would be awesome! Region grouping or other related proposals for efficient wal tailing deserves it's own design doc(s). > > > > I think that would be great. Back when we did snapshots, we had active > development against a prototype and spent a bit of time breaking it down > into manageable more polished pieces that had slightly lenient reviews. > This exercise really helped us with our interfaces. We committed code to > the dev branch which limited merge pains and diff for modifications made by > different contributors. In the end when we had something we were happy > with on the dev branch we merged with trunk and fixed bugs/diffs that > cropped up in the mean time. I'd suggest a similar process for this. > Agreed. We can make use of the previous best practices. Shame that we still do not have read-write git repo. > > > -- > // Jonathan Hsieh (shay) > // Software Engineer, Cloudera > // j...@cloudera.com >
