On Wed, Jan 15, 2014 at 3:57 PM, Enis Söztutar <enis....@gmail.com> wrote:
> I am afraid, it is not coprocessors or current set of plugins only. We need > changes in the > RPC, meta, region server, LB and master. Since we cannot easily get hooks > into all these in > a clean manner, implementing this purely outside would be next to > impossible. > I'm pretty unconvinced that this is the correct way forward. It seems to introduce a lot of risk without a lot of gain. Right now to me the 100% correct way forward is through paxos. That's a lot of work but it has the most payoff in the end. It will allow much faster recovery, much easier read sharding, it allows the greatest flexibility on IO. On the other end of the spectrum is something like MySQL/Postgres read slaves (either tables or clusters). Read slaves built on top of what's currently there seem to give all of the benefits of read slaves built into the current HBase without all of the risk. Sharding on top of the already built datastore is a pretty well known and well understood problem. There are lots of great example of making this scale to pretty insane heights. You lose very little flexibility and incur almost not risk to the stability of HBase.
